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PREFACE 


This  technical  report  was  prepared  as  an  in-house  effort  within  the 
RSM  Engineering  Techniques  Section  (RBRT)  of  the  Reliability  Branch  (RBR) 
and  the  Reliability  and  Compatibility  Division  (RB),  respectively,  at  RADC. 

The  study  was  done  in  two  separate  calendar  time  periods.  Early  efforts 
were  concerned  with  investigations  into  the  details  of  the  field  maintenance 
data  collection  systems,  the  procedures  to  be  used  for  the  data  compilation 
and  preliminary  analyses  of  the  data.  The  results  of  this  early  work,  which 
are  covered  mainly  in  Sections  1-3  of  the  report,  were  presented  at  that 
time  in  various  briefings  at  Hq  AFSC  and  at  RADC.  These  early  efforts  also 
provided  some  of  the  groundwork  for  additional  contractual  studies  into  the 
field  operational  influences  on  the  reliability  and  maintainability  of 
electronic  equipment. 

The  second  phase  of  the  work,  which  is  covered  in  Sections  4-7  of  the 
report,  was  done  more  recently  after  more  detailed  analyses  of  the  field 
data  were  done.  Hopefully,  the  conclusions  and  recomnendations  made  in  the 
report  will  foster  improvements  in  the  field  data,  as  well  as  provide 
guidance  to  other  DoD  agencies  and/or  contractors  who  may  be  faced  with 
using  field  maintenance  data  for  R&M  assessments. 

Acknowledgement  is  given  to  Mr.  George  Lyne  (RBRT)  for  his  assistance 
in  preparing  some  of  the  computer  plots  contained  in  the  report, 

Mr.  Irvin  Krulac  of  the  Reliability  Analysis  Center  (RBRAC)  for  his 
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assistance  in  providing  details  regarding  the  field  data  collection 
forms,  and  to  Mrs.  Johanna  Leonard  and  Mrs.  Patricia  Parkhurst  for  their 
typing  and  chart  preparation  assistance. 

The  helpful  suggestions  provided  by  Messrs.  Anthony  Coppola  and 
Lester  0.  Gubbins  of  RBRT  are  also  appreciated. 
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Reliability  and  Maintainability  (R&M)  field  performance  data  obtained 
from  Air  Force  field  mainentance  data  reporting  systems,  represent  a 
potentially  very  valuable  source  of  information  for  R&M  technique  develop- 
ment and  improvement.  The  extremely  large  samples  of  "real  world"  R&M  data, 
which  result  from  field  operation,  are  not  available  from  any  other  source. 
Field  R&M  data  analyses,  grounded  upon  good  quality  field  data,  can  provide 
the  bases  for  evaluating  the  effectiveness  of  the  R&M  assurance  efforts  that 
went  into  the  design,  testing  and  screening  of  the  equipment  and  its 
components.  Good  quality  field  data  not  only  can  be  used  to  improve  R&M 
assurance  techniques  for  new  systems  programs,  but  can  also  be  used  for 
improving  the  R&M  and  lowering  the  support  costs  of  currently  fielded 
equipment. 

Air  Force  maintenance  and  equipment  status  reporting  systems  were 
designed  primarily  for  logistics,  maintenance,  and  inventory  management; 
rather  than  specifically  for  field  R&M  analysis  purposes.  Attempts  at 
using  the  field  data  for  R&M  analyses  have  been  the  source  of  much  concern 
regarding  the  quality  and  usability  of  the  data.  Recently,  there  have  been 
improvements  made  in  the  data  systems  as  a result  of  the  efforts  of  special 
groups  which  were  established  to  address  the  field  R&M  data  improvement 
issue.  There  remains,  however,  a need  for  good  quality  field  R&M  data  which 
can  be  used  for  fully  achieving  field  R&M  assessment  objectives. 

This  study  was  undertaken  to  determine  the  extent  to  which  Air  Force 
maintenance  data  systems  can  be  used  for  achieving  specifically  defined  R&M 
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analysis  objectives.  Various  comparisons  of  field  derived  R&M  estimates  at 
the  part,  board  and  equipment  levels,  versus  corresponding  estimates  derived 
during  the  equipment  R&M  development  program,  were  made  in  the  study  for  a 
sample  ground  electronic  system.  In  attempting  to  perform  such  analyses,  the 
limitations  of  the  current  data  systems  were  pointed  out  and  recommendations 
for  improvement  were  developed. 

The  study  has  shown  that  much  of  the  data  needed  for  field  R&M  assess- 
ments are  already  contained  in  the  current  data  systems  or  can  be  obtained 
in  combination  with  associated  maintenance  documentation.  In  fact,  the 
levels  of  analysis  detail  attained  through  the  study  go  much  further  than 
was  formerly  thought  possible.  The  data  systems  fall  short  in  important 
areas,  however,  which  severly  limit  their  use  for  fully  achieving  R&M 
analysis  objectives.  It  was  found  that,  although  field  data  collection 
procedural  improvements  are  needed,  the  most  important  improvements  required 
fall  outside  the  scope  of  field  maintenance  data  system  procedures.  Some  of 
the  key  data  required  to  perform  field  reliability  assessments  (e.g., 
knowledge  of  root  causes  of  failure)  cannot  be  provided  by  field  maintenance 
personnel.  These  and  other  inputs  to  field  R&M  assessment  efforts  must  be 
developed  by  the  group  or  agency  responsible  for  the  R&M  data  collection  and 
analyses. 

The  findings  of  the  study  and  the  data  compilation  procedures  discussed 
in  the  body  of  the  report  can  be  used  for  guidance  In  performing  similar 
field  R&M  assessments  for  other  ground  electronic  equipments.  Recommendations 
for  improving  the  quality  and  usability  of  R&M  data,  collected  from  the 
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field,  are  made  in  the  report.  It  is  believed  that  implementation  of  the 
recommendations  by  both  the  R&M  groups  responsible  for  the  data  collection 


and  analysis,  as  well  as  by  cognizant  Air  Force  agencies,  will  provide  for 
further  developments  and  use  of  this  very  valuable  source  of  R&M  data. 
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1.0  INTRODUCTION 

Analysis  of  the  Reliability  and  Maintainability  (R&M)  characteristics 
of  military  electronic  equipment  in  the  field  use  environment  can  be  a very 
useful  tool  for  R&M  techniques  development  and  applied  research.  When  good 
quality  field  data  are  available,  then  the  field  derived  R&M  estimates 
represent  the  final  proof  of  the  effectiveness  of  all  of  the  R&M  assurance 
efforts  that  went  into  the  design,  testing,  and  screening  of  the  equipment 
and  its  components.  There  are  many  reasons  why  detailed  R&M  analyses  of 
field  performance  data  should  be  done.  Given  reasonably  complete  and 
accurate  field  data,  some  of  the  objectives  and  benefits  which  can  be 
derived  in  performing  such  analyses  are: 

(a)  Correlation  of  R&M  design  prediction  and  laboratory  demonstration/ 
test  data  with  field  results  can  be  performed  to  assess  and  identify  the 
need  for  improvements  in  these  R&M  assurance  and  control  techniques. 

(b)  R&M  problem  areas  can  be  isolated  and  corrective  actions  can  be 
taken  to  improve  equipment  performance  and  reduce  support  costs  in  the  field. 

(c)  Realistic  R&M  requirements  and  engineering  data  can  be  developed 
for  application  to  new  system  programs. 

(d)  Field  experience  histories  for  a variety  of  equipments  can  be 
retained  in  a data  bank  for  use  in  the  development  of  improved  R&M 
techniques  which  more  accurately  reflect  the  field  use  environment. 
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(e)  Field  failure  and  repair-time  statistical  distributions  can  be 
analyzed  for  compatibility  with  the  distribution  assumptions  which  were 


used  during  the  design  and  development  stages. 


(f)  Part  failure  rate  estimates  can  be  derived  from  the  field  data 
based  upon  large  data  samples,  otherwise  not  obtainable,  which  can  be  used 
to  verify  and/or  modify  MIL-Handbook  failure  rate  values  used  in  the 
reliability  predictions. 

(g)  Evaluation  of  "lessons-learned",  and  the  need  for  improvements 
in  various  elements  of  the  R&M  program  which  was  conducted  during  design, 
development  and  production  ( i . e . , part/assembly  screening,  growth  tests, 
environmental  tests,  etc.)  can  be  made. 


(h)  Field  R&M  performance  can  be  monitored  to  determine  if  degradation 
has  occurred  due  to  faulty  maintenance  practices,  changes  in  the  maintenance 
concept,  or  to  inadequate  replacement  spares  procurement  programs. 

R&M  analyses  of  field  data,  which  have  as  their  objectives  one  or  more 
of  those  listed  above,  require  good  quality  field  data.  Unfortunately,  the 
Air  Force  maintenance  data  systems  which  are  currently  in  use  for  reporting 
and  processing  field  malfunction  data  have  not  been  designed  with  R&M 
analyses  as  a primary  objective.  As  a result,  needed  data  elements  are  not 
readily  available  and  the  output  products  from  the  data  systems  are  not 
prepared  in  a manner  which  facilitates  R6M  analyses.  Traceability  of  equip- 
ment failures  to  causes  at  lower  assembly  levels  is  not  always  possible. 

Part  and  assembly  identifications  between  corresponding  hardware  elements 
of  the  design  prediction  and  the  field  failure  data  also  present  a per- 
plexing problem.  In  addition,  errors  and  omissions  in  the  field  reported 
data  contribute  to  the  difficulties  in  performing  meaningful  R&M  analyses. 
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Nonetheless,  the  limitations  in  the  Air  Force  data  systems  may  be 


compensated  for  in  some  respects.  Useful  analyses  can  perhaps  be 
performed  by:  taking  advantage  of  the  large  quantities  of  data  which  are 
available;  using  supplemental  information  (parts  identification  lists, 
technical  manuals,  etc.)  to  aid  in  resolving  hardware  identification 
ambiguities;  combining  field  data  which  are  collected  jointly  under  separate 
data  systems  and  exercising  necessary  caution  either  in  interpreting  or  by 
censoring  any  obviously  questionable  or  incomplete  data.  The  problem  then 
is  to  determine  the  extent  to  which  Air  Force  maintenance  data  can  be  used 
for  field  R&M  assessment  purposes  and  to  develop  recommendations  for  improv- 
ing the  data  systems  and  procedures  for  use  in  field  R&M  evaluations. 


Objective  and  Approach 


The  objective  of  this  study  was  to  use  the  information  which  is 
available  from  Air  Force  maintenance  data  systems,  and  other  related  sources, 
to  the  maximum  extent  possible,  for  the  analysis  of  the  R&M  characteristics 
of  a fielded  ground  electronic  equipment.  Since  the  data  available  for 
analyses  are  known  to  be  deficient  in  some  respects,  analytical  objectives 
outlined  In  the  previous  introductory  section  were  adhered  to  as  general 
goals  rather  than  as  firm  objectives. 


The  approach  for  the  study  was  to  use,  as  a study  vehicle,  a 
diversely  deployed,  ground-based  electronic  equipment  for  which  R&M  program 
and  field  data  were  readily  available.  Standard  Air  Force  field  maintenance 
data,  collected  from  various  field  operating  sites,  were  used  In  the  RSM 
analyses.  Failure  rate  estimates  obtained  during  the  design  and  development 


stages,  at  the  part,  subassembly  and  system  levels  were  compared  against 
similar  estimates  derived  from  the  field  maintenance  data.  In  addition, 
verification  and  evaluation  of  the  failure  and  repair-time  statistical 
distribution  assumptions  used  during  the  design  and  development  stages,  with 
field  experience  data,  were  also  made.  Thus,  in  attempting  to  perform  such 
analyses,  the  limitations  in  using  current  Air  Force  maintenance  data  systems 
for  the  R&M  analysis  objectives  outlined  in  the  introduction  of  the  report, 
were  pointed  out  and  recommendations  for  improvements  were  developed. 

1.2  Air  Force  Maintenance  Data  System  Overview 

The  Maintenance  Data  Collection  (MDC)  System  established  by  the 
Air  Force  Manual  66-1,  "Maintenance  Management",  and  the  "Standard  Aerospace 
Vehicle  and  Equipment  Status  Report",  established  by  AF  Manual  65-110  are 
the  prime 'y  sources  of  R&M  field  data  in  the  Air  Force.  These  manuals,  and 
their  associated  documentation,  provide  the  detailed  instructions  for 
completing  the  reporting  forms  (AFTO  349  and  350),  (AF  Forms  182  or  2445) 
by  the  Air  Force  field  technicians  and  for  the  subsequent  processing  of  the 
data.  The  completed  data  forms  are  compiled  at  central  data  processing 
facilities  which  provide  editing,  sorting,  summary  report  preparation  and 
distribution  functions.  The  computer  processed  output  products  from  these 
data  systems  were  the  source  of  the  R&M  field  data  on  which  the  analyses  in 
this  report  were  based.  The  following  is  a brief  description  of  each  of  the 
data  systems. 

The  MDC  system  was  designed  primarily  as  a base  level  maintenance 
management  system.  The  objectives  of  the  system  are  to  provide  maintenance 


managers  with  information  on  the  maintenance  accomplished  by  assigned 
personnel,  to  identify  the  equipment  on  which  work  was  accomplished,  to 
identify  the  reasons  why  the  work  was  required  and  the  actions  required  to 
complete  the  maintenance  job.  All  maintenance  actions  involving  direct  labor 
expenditure  such  as  scheduled  inspections,  preventive  maintenance,  and  un- 
scheduled maintenance,  both  on-line  and  off-line,  are  reported  in  the  MDC 
system.  Data  on  material  consumption  are  also  reported  to  facilitate  spares 
management. 

The  AFM  65-110  Equipment  Status  Reporting  System  (ESR)  was  designed 
to  provide  three  separate  reporting  functions,  which  are: 

(a)  Inventory  Management:  The  quantities  of  specific  equipment 
types  which  are  assigned  to  a given  organization  are  reported  to  the  managing 
activity.  Inventory  records  are  maintained  by  equipment  serial  number  and 
are  periodically  changed  by  the  reporting  of  gain,  loss  or  termination 
transactions. 

(b)  Equipment  Status:  Equipment  status  monitoring  is 
accomplished  by  the  reporting  of  "Not  Operationally  Ready"  (NOR)  conditions 
for  each  fielded  equipment.  For  example,  when  the  equipment  is  down  for 
maintenance,  the  start  and  stop  times  for  the  downtime  are  reported  and  the 
malfunctioning  item  responsible  for  the  (NORM)  condition  is  identified.  In 
addition,  downtime  is  also  reported  for  conditions  other  than  maintenance 
(i.e.,  supply  delays,  training,  etc.). 

(c)  Equipment  Utilization:  The  utilization  of  the  equipment 
in  terms  of  mission  type,  elapsed  flying  time,  etc.,  are  reported  for 


avionic  type  equipment.  For  ground  electronic  equipment,  as  is  the  case  for 
the  equipment  selected  for  use  in  this  study,  the  reporting  is  on  the  basis 


of  exception  to  normal  conditions.  For  example,  since  the  mission  of  the 
equipment  requires  24  hour  per  day  operation,  the  equipment  is  considered 
operating  unless  a form  is  submitted  indicating  that  the  equipment  is  down 
for  maintenance  or  for  other  reasons. 

Details  of  the  MDC  and  AFM  65-110  systems,  including  copies  of  the 
forms  used  (AFTO  349  and  AF  Form  182)  and  the  data  elements  contained  on 
the  forms  are  provided  in  Appendix  A.  A comparison  of  the  data  elements 
contained  on  the  forms  against  a composite  list  of  elements  needed  for  R&M 
assessment  is  made  in  Appendix  B. 


1.3  Equipment  Selected  for  Use  in  the  Stud 


The  equipment  used  for  the  R&M  analyses  is  part  of  an  advanced, 
ground-based,  digital  processing  system  which  is  used  by  the  Air  Force  and 
the  Federal  Aviation  Administration  in  support  of  air  traffic  control  and 
surveillance  requirements.  The  equipment  was  initially  fielded  in  1972  and 
*s  identified  by  the  Air  Force  nomenclature  AN/FYQ-47.  Briefly,  the  function 
of  the  AN/FYQ-47  is  to  receive  raw  radar  and  beacon  inputs  from  associated 
equipment,  reject  data  caused  by  weather,  ground  clutter  and  noise,  and 
perform  statistical  detection  to  determine  the  presence  of  aircraft.  The 
digital  data,  derived  from  the  processing  of  the  raw  input  information,  are 
inserted  into  appropriately  formatted  standard  messages  which  are  transmitted 
over  telephone  lines  to  central  processing  locations. 


From  a R&M  viewpoint,  the  AN/FYQ-47  is  comprised  of  approximately 


17,000  electronic  parts,  6,250  of  which  are  monolithic  microcircuits, 

2,400  transistors  and  diodes,  and  over  8,000  capacitors  and  resistors. 
Practically  all  of  the  electronic  components  are  contained  on  808  plug-in 
circuit  boards  which  form  the  basic  on-line  replaceable  units  in  the  event 
of  malfunction.  The  mechanical  and  electromechanical  components  of  the 
AN/FYQ-47  represent  a very  small  proportion  of  the  overall  part  complement 
and  include  servo-mechanisms,  gear  trains,  switches,  and  tracking  hardware. 

Corrective  maintenance  on  the  AN/FYQ-47  is  facilitated  by  the  use 
of  built-in-test  equipment  which  provides  for  fault  localization  to  small 
groups  of  replaceable  plug-in  circuit  boards.  Removal  and  replacement  of 
the  circuit  boards  must  be  performed  in  a specific  sequence  until  it  is 
determined,  via  appropriate  technical  manual  test  procedures,  that  the 
failed  condition  has  been  corrected.  Removed  circuit  boards  are  then  checked 
on  an  off-line  card  tester  (AN/FYM-27)  where  fault  isolation  to  piece-parts 
(capacitors,  microcircuits,  transistors,  etc.)  and  subsequent  repairs  of  the 
boards  are  accomplished.  Repaired  circuit  boards  are  returned  to  spares 
stock.  Some  of  the  equipment  (e.g.,  power  supplies)  contain  hard-wired 
components  which  must  be  replaced  on-line  in  the  event  of  failure. 


2.0  RELIABILITY  AND  MAINTAINABILITY  PROGRAM  DATA 
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During  the  design,  development  and  production  of  the  AN/FYQ-47,  an  R&M 
program  was  conducted,  in  accordance  with  Mil-Standards  785^ ) and  470^, 
to  provide  the  necessary  controls  and  assurances  that  the  contractually 
specified  R&M  requirements  for  the  equipment  could  be  achieved.  As  part  of 
the  program,  R&M  design  predictions  were  developed  to  assess  if  the  equipment 
design,  i.e.,  quality  and  stress  levels  of  the  parts  used,  use  of  built-in- 
test  equipment,  modularity,  redundancy,  etc.,  result  in  an  equipment  capable 
of  meeting  the  specified  R&M  requirements.  In  addition,  controlled  and 
monitored  R&M  demonstration  tests  were  conducted  to  formally  demonstrate 
that  the  equipment  could  meet  or  better  the  specified  R&M  requirements. 

Other  reliability  assurance  efforts,  such  as  parts  screening  and  equipment 
burn-in  testing,  were  also  conducted  as  part  of  the  reliability  program. 

The  following  sections  provide  a brief  description  and  the  results  of 
each  of  the  R&M  program  efforts.  Since  the  reliability  prediction  and 
demonstration  mathematical  models  are  to  be  referred  to  in  the  field  data 
analyses  sections  of  the  report,  a brief  discussion  of  some  of  the  assumptions 
underlying  these  models  is  also  presented. 

2.1  Reliability  Design  Prediction 

The  contractually  specified  reliability  requirement  for  the  AN/FYQ-47 
was  a MTBF  of  545  hours.  A reliability  prediction  was  performed  for  the 


(1)  MIL- STD- 785,  "Reliability  Program  for  Systems  & Equipment  Development 
and  Production",  28  Mar  69. 

(2)  MIL-STD-470,  "Maintainability  Program  Requirements  for  Systems  and 
Equipments",  21  Mar  66. 
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equipment  using  failure  rates  contained  in  the  RADC  Reliability  Notebook^). 
The  current  MIL-HDBK-217B^  has  since  replaced  the  former  Notebook. 

Figure  1 below,  shows  the  reliability  model  for  the  AN/FYQ-47  and 
the  respective  predicted  failure  rates  for  various  functional  groups  which 
make  up  the  equipment.  Both  a series  and  parallel  structure  describe  the 
reliability  operation  of  the  equipment. 


HEIGHT 
SHELF  I 
HF1 


ARTG/TTG.  TOG/MOG 
OBG/DTG/TPG.MCG/ 
MEM.  TCU/GFM 


X • 66.63 
(FOR  EACH  SHELF) 


X,  * 1126.84 


X2  * 44.75 


X.  - 447.04 


HEIGHT 
SHELF  2 
HFO 


HEIGHT 
SHELF  2 
HF1 


X,  4 x2  ♦ x3  ♦ x4  4 X£  - xT 
•ALL  FAILURE  RATES  XX  10'6  HOURS 


XT  * 1727.88 


MTBF  - 578.7  HOURS  • t 


FIGURE  1.  AN/FYQ-47  RELIABILITY  PREDICTION  MODEL 


As  can  be  noted,  the  total  predicted  failure  rate  for  the  AN/FYQ-47 
is  dominated  by  the  series  elements.  The  reliability  prediction  model 


T3)  "RADC  Reliability  Notebook",  Vol  II,  RADC-TR-67-108,  Nov  67. 

(4)  MIL-HDBK-217B,  "Reliability  Prediction  of  Electronic  Equipment", 
20  Sep  74.(821  640) 
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includes  calculations  for  redundancy  with  repair  and,  as  shown  by  the 
equivalent  failure  rate  X£  of  .0089X10"^,  the  system  failure  rate  contribution 
of  the  parallel  structure  is  negligible.  The  discussion  below,  therefore, 
regarding  the  assumptions  underlying  the  reliability  prediction  model,  is 
given  in  a series  context. 

In  a series  configuration,  the  successful  operation  of  the  system 
depends  upon  the  failure- free  operation,  for  some  time  (t),  of  all  components 
in  the  series  structure.  The  reliability  of  a series  structure  is  given  by 
the  expression: 

R(t)  = p(x , ) p(x2|x1)  p(x3|x1x2)  •••  P(xn|x1x2  ...  Xn_-| ) 

where: 


R(t)  = the  system  reliability 

X.j  are  the  series  connected  components 

and: 

P ( X^  | X-| X2  ...  X.j  i ) are  conditional  survival  probabilities. 

If  the  n series  connected  components  X-j , X2,  ...  Xn  are  independent,  then: 

R(t)  = P(X3)  P(X2)  P(X3)  •••  P(Xn)  = P(Xi ) 

If  also,  in  addition,  each  component  X^  has  a constant  failure  rate  Xi , then 
the  exponential  failure  density  function  is  appropriate  so  that: 

n . t n 

R(t)  = n e 1 = exp  ( - E X-t) 

i=l  i=l 
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In  order  for  the  former  exponential  model  to  hold  then,  the 
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following  must  be  generally  true: 


(a)  The  system  reliability  configuration  must  be  in  series. 

(b)  The  individual  components  must  have  a constant  failure 

rate. 

(c)  The  individual  components  must  be,  both  in  a functional 
and  statistical  sense,  independent. 

It  should  be  noted,  however,  that  although  the  individual  component 
failures  may  be  representative  of  non-constant  failure  rate  processes,  it 
has  been  shown  that  if  a large  number  of  such  processes  are  pooled,  then  the 
distribution  of  the  times  between  failures  for  the  system  approaches  the 
exponential . 

Under  the  assumptions  discussed  in  the  previous  paragraph,  and 
using  the  reliability  model  depicted  in  Figure  1,  the  predicted  MTBF  for 
the  AN/FYQ-47  was  determined  to  be  578.7  hours.  The  predicted  MTBF  value 
can  be  interpreted  as  the  inherent  MTBF  of  the  equipment,  given  that  the 
part  quality  levels  and  the  design  were  not  compromised  by  poor  workmanship, 
process  or  quality  controls  during  the  manufacture  of  the  equipment  and  its 
components.  The  predicted  MTBF  of  578.7  hours  thus  indicated  that  the 
equipment  design,  potentially  at  least,  was  capable  of  meeting  the  contract- 
ually specified  545  hour  MTBF  requirement. 

2.2  Reliability  Demonstration 


The  AN/FYQ-47  was  subjected  to  two  reliability  demonstration  tests. 


During  the  development  program,  one  equipment  was  operated  continuously  for 
1,271  hours  without  any  system  outages.  Failures  occurred  in  the  redundant 
height  shelves,  but  at  no  time  were  both  shelves  inoperative  resulting  in  a 
total  system  outage.  A second  controlled  test  was  also  performed  on  one 

( 5) 

production  system.  The  test  was  a sequential  test  similar  to  MI L-STD- 781 v , 
Test  Plan  V.  This  test  plan  is  based  upon  the  assumption  that  the  failure 
rates  are  constant  and  that  the  exponential  failure  model  is  applicable. 
During  the  test,  the  equipment  was  operated  continuously  and  performance 
was  checked  three  times  a day.  The  test  was  terminated  with  an  accept 
decision  with  one  relevant  failure  at  1,650  hours  of  operation.  Given  that 
the  exponential  failure  model  is  applicable,  the  lower  90%  confidence  limit 
on  the  MTBF  is  given  by: 

A 

9 = JL. 

L x2  (a,  2r) 

where 

r = the  number  of  failures 
T = operating  time 
a = 10% 

~y2=  Chi-Square  variable  with  2r  degrees  of  freedom 
or 

0L  = 2 0 650l  = 3300  = 715>8  hours 
x2(.l,2)  4.61 

Thus,  the  demonstration  results  indicate  that  the  sample  production 
IF)  MIL-STD-781,  "Reliability  Tests,  Exponential  Distribution",  15  Nov  67. 
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equipment  surpasses  the  545  hour  MTBF  requirement.  Obviously,  assessments 
as  to  whether  the  remaining  production  equipments  have  met  the  requirement 
can  only  be  made  from  analyses  of  the  field  data. 

2.3  Parts  Control,  Screening,  Equipment  Burn-In 

All  monolithic  microci rcui ts  contained  in  the  equipment  were  sub- 
jected to  quality  assurance  screening  tests  per  RADC  Specification  2867A^6^, 
the  forerunner  to  the  present  MIL-STD-883^.  Controls  by  production  lot 
and  device  type  were  implemented  to  ensure  identification  of  bad  lots  and 
corrective  actions  where  required.  Part  screening  tests  included  precap 
visual  inspection,  thermal  shock,  centrifuge,  hermeticity  and  burn-in  of 
240  hours  at  125°C.  Limit  tests  on  samples  of  various  production  lots  were 
also  performed  to  establish  the  limits  of  the  stress  which  the  microcircuits 
could  withstand  and  to  identify  the  failure  modes  involved. 

In  addition  to  the  100%  screening  of  all  microcircuits,  a controlled 
100  hour  system  burn-in  test  at  the  factory  was  performed  prior  to  shipping 
each  system  to  the  field.  The  purpose  of  the  burn-in  test  was  to  weed-out, 
identify,  and  correct  in  subsequent  production  units,  early  failure  patterns 
(workmanship,  substandard  parts,  etc.)  in  the  equipment. 


£ 

Ll 


2.4  M Design  Prediction 

The  contractually  specified  maintainability  requirement  for  the 
AN/FYQ-47  was  a mean-time- to- repair  of  .5  hours  and  a maximum  time- to- repair 
(98th  percentile)  of  2 hours.  A maintainability  prediction  for  the  equipment 


16)1  RADC  Specification  2867A,  "Quality  and  Reliability  Assurance 
Procedures  for  Monolithic  Microcircuits",  30  Jan  67. 

(7)  MIL-STD-883,  "Test  Methods  & Procedures  for  Microelectronics",  15  Nov  74. 
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was  performed  using  Procedure  II  of  MIL-HDBK-472(®).  The  maintenance  design 


r 


l.* 

► 


concept  on  which  the  prediction  was  based  included: 


(a)  Automatic  fault  localization  via  built-in-test  with  the 
occurrence  of  a fault  made  known  by  audio  or  video  alarm. 


(b)  Fault  isolation  to  a specific  board  by  use  of  fault 
diagnosis  and  specific  board  replacement  schemes  contained  in  the  maintenance 
procedures. 

(c)  Replacement  of  the  defective  board  by  an  operational  spare 
and  subsequent  off-line  repair  of  the  board  with  the  aid  of  a board  tester. 
Off-line  repair  time  was,  of  course,  not  included  in  the  maintainability 
prediction. 

Based  upon  the  former  maintenance  design  concept,  and  use  of 
Procedure  II,  the  predicted  mean-time-to-repair  (MTTR)  for  the  AN/FYQ-47  was 
determined  to  be  24.8  minutes.  Again,  this  value  can  be  interpreted  as  the 
inherent  "designed-in"  MTTR  of  the  equipment.  It  should  be  noted,  however, 
that  Procedure  II  relies  rather  heavily  on  subjective  estimates  of  repair 
times  by  the  maintainability  analyst. 


2.5  M Demonstration 


A maintainability  demonstration  test  was  conducted  at  the  contractor's 
plant  in  accordance  with  MIL- STD-471 Test  Method  1.  Fifty  (50) 

T5]  MIL-HDBK-472,  "Maintainability  Prediction",  24  May  66. 

(9)  MIL-STD-471,  "Maintainability  Demonstration",  27  Mar  70. 
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maintenance  tasks  were  randomly  selected  and  faults  were  inserted  into  the 
equipment  to  simulate  equipment  failures.  Air  Force  and  FAA  technician 
personnel  were  used  in  the  demonstration  to  perform  the  repair  actions  for 
the  simulated  failures.  Based  upon  actual  timings  of  the  50  maintenance 
tasks  used  in  the  demonstration,  a MTTR  of  17.5  minutes  was  demonstrated, 
with  98%  of  the  maintenance  tasks  being  accomplished  in  less  than  two  hours. 
The  demonstration  results  thus  surpassed  the  specified  maintainability 
requirements.  However,  it  is  well  known  that  demonstrations,  in  general, 
are  conducted  under  ideal  conditions.  Maintenance  in  the  field  includes 
many  operational  influences  which  are  not  and  cannot  normally  be  accounted 
for  in  the  demonstration.  Analyses  of  field  repair  time  data  can,  however, 
provide  a means  of  evaluating  the  effectiveness  of  the  procedures  and  the 
accuracy  of  the  estimates  derived  from  the  maintainability  prediction  and 
demonstration  techniques. 


3.0  FIELD  DATA  COMPILATION  AND  PRELIMINARY  ANALYSES 


Prior  to  obtaining  the  data  to  be  used  for  the  analyses,  a review  of  the 
standard  Air  Force  maintenance  data  collection  and  processing  procedures 
was  initially  made.  As  a result  of  the  review,  it  was  found  that  the  infor- 
mation required  to  perform  the  analysis  was  not  provided  completely  in  any 
single  standard  computer  output  data  product.  It  was  therefore,  necessary 
to  obtain  the  data  from  several  sources,  have  it  machine  processed  in 
a special  chronological  manner  so  that  the  information  could  be  combined  and 
compiled  for  the  analyses. 

Figure  2 illustrates  the  various  elements  of  the  data  reduction  and  study 
plan.  The  raw  data  sources  in  the  field  are  shown  as  the  Maintenance  Data 
Collection  Record  (MDC)  (AFTO  Form  349)  and  the  Equipment  Status  Report  (ESR) 
(AF  Form  182).  Details  regarding  the  data  elements  contained  on  these  forms 
are  provided  in  Appendix  A.  As  was  discussed  previously  in  Section  1,  the 
MDC  system  contains  information  pertaining  basically  to  resource  expenditures 
of  manpower  and  materials  for  corrective  and  preventive  maintenance.  The 
ESR  data  provides  information  on  the  equipment  operating  status  and  utiliza- 
tion in  terms  of  downtime  for  maintenance,  training,  supply  delays,  etc.  By 
combining  these  data  sources,  it  was  possible  to  obtain  many  of  the  data 
elements  needed  for  the  R&M  analyses,  although  with  some  missing  detail. 

Figure  2 also  illustrated  that  it  was  necessary  to  use  the  maintenance 
and  work  unit  code  manuals,  the  servicing  and  inspection  instructions,  the 
illustrated  parts  breakdown  and  parts  cross-identification  lists  for  the 
equipment  selected  for  use  in  the  study. 
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FIGURE  2.  DATA  COMPILATION  AND  STUDY  PLAN 
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The  former  documents  served  several  purposes.  Some  of  the  discrepancies  in 


the  field  data,  such  as  failed  board/part  identification  ambiguities  or 

•j 

incomplete  information  were  resolved  by  using  the  maintenance  manuals.  The 
servicing  instructions  and  inspection  requirements  permitted  identification 
of  the  type  of  maintenance  actions  which  were  performed  during  preventive 
maintenance.  The  illustrated  parts  breakdown  and  part  cross-identification 
lists  provided  a means  for  counting  the  parts  in  the  equipment  and  for 
resolving  part  identification  problems  between  corresponding  hardware 
elements  of  the  prediction  and  field  failure  data.  Work  unit  code  manuals 
were  used  to  identify  failed  modules,  assemblies,  etc.,  in  the  field  data 
and  to  correlate  failed  hardware  identification  with  the  technical  manuals 
and  the  prediction  data. 

3.1  Preparation  of  Equipment  R&M  Logs 

Since  the  data  pertinent  to  the  analysis  was  contained  in  both 
the  AFM  66-1  and  AFM  65-110  data  sources,  as  well  as  in  the  supplementary 
information  provided  by  the  technical  manuals,  parts  list,  etc.,  it  was 
necessary  to  consolidate  the  data  into  a single  source. 

The  equipment  R&M  Log  shown  in  Fioure  2 was  established  for 
this  purpose.  A sample  page  of  the  log  is  provided  in  Figure  3. 


J 


FIGURE  3.  SAMPLE  EQUIPMENT  R&M  LOG 


Using  a chronological  sort  of  the  data,  and  by  matching  either  calendar  dates 
or  the  Job  Control  Number  (JCN)  in  the  AFM  66-1  data  with  the  Equipment 
Status  Report  Number  (ESR)  in  AFM  65-110  data,  it  was  possible  to  identify 
for  many  of  the  malfunctions  and  for  each  equipment: 

(a)  The  item  or  items  which  failed  or  were  replaced  (Work  Unit 
Code  (WUC)). 

(b)  Whether  the  malfunction  caused  partial  or  total  system  outage 
(RG/RN)'  or  (AG/AN). 

(c)  The  parts  replaced  as  the  result  of  the  malfunction  (Federal 
Stock  Code,  Part  Number,  Reference  Designator). 
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(d)  The  operating  time  between  malfunctions. 


(e)  The  type  of  maintenance  action  (preventive  or  corrective). 

(f)  Maintenance  downtime,  supply  and  administrative  delay  times. 

(g)  Equipment  operating  time. 

(h)  Coded  information  pertaining  to  when  the  malfunction  was 
discovered  (WD),  how  the  item  malfunctioned  (HM),  and  the  action  taken  for 
the  repair  (AT). 

Operating  time  for  each  equipment  was  obtained  by  subtracting  the 
maintenance  and  delay  downtimes  from  the  total  available  calendar  time. 

Since  the  mission  of  the  equipment  requires  24  hour  per  day  operation,  the 
equipment  was  considered  operating  unless  the  data  indicated  it  was  down 
for  maintenance,  logistic  or  administrative  delay.  It  should  be  noted, 
however,  that  it  was  not  generally  possible  to  positively  identify  the 
primary  hardware  cause  of  the  system  malfunction.  Omissions  in  the  data 
and  maintenance  actions  involving  multiple  board  or  part  replacements  generally 
prevented  such  identification.  In  addition,  it  should  be  recognized  that 
the  Air  Force  field  technician  is  primarily  motivated  to  "fix"  the  equipment 
and  not  to  isolate  failure  causes.  As  a result,  the  field  data  tend  to 
reflect  replacement  actions  which  were  performed  to  facilitate  the  repair 
rather  than  specifics  regarding  the  cause  of  failure. 

3.2  Study  Plan 

Figure  2 on  page  17  also  illustrates  the  sequence  of  the  various 
data  compilation  and  analysis  tasks  of  the  study  plan. 


The  Equipment  R&M  Logs  were  initially  compiled  using  one  year  of 
field  operational  data  on  31  equipments  which  were  located  at  different 
Air  Force  sites  within  the  United  States.  As  shown  in  Figure  2,  the  data 
were  used  in  a preliminary  analyses  contained  in  Section  3.4  of  the  report, 
to  determine  if  the  data  were  of  sufficiently  good  quality  to  attempt 
performing  further  analyses.  The  preliminary  analysis  indicated  that  the 
data  were  reasonably  usable  for  more  detailed  part  level  and  failure/repair- 
time distribution  analyses.  Comparisons  of  predicted  versus  observed  failure 
rates  are  made  in  Section  4 of  the  report  for  the  parts  and  circuit  boards 
contained  in  the  equipment.  An  additional  nine  months  of  field  operation 
data  were  compiled  for  25  of  the  31  equipments,  which  were  combined  with  the 
data  collected  earlier  for  the  preliminary  analyses.  Using  the  additional 
data,  failure  distribution  analyses,  including  trend  tests  and  goodness-of- 
fit  tests,  are  performed  in  Section  5 and  comparisons  with  prediction  and 
demonstration  statistical  distribution  assumptions  are  made.  The  field 
maintainability  characteristics  of  the  equipment  are  evaluated  in  Section  6 
of  the  report.  Conclusions  and  recommendations  are  presented  in  Section  7. 


3.3  Failure/Malfunction  Classifications 


In  the  process  of  compiling  the  Equipment  R&M  Logs,  it  was  found 


that  both  the  AFM  66-1  and  AFM  65-110  data  records  were  incomplete.  For 


example,  it  was  possible  to  match  the  Job  Control  Number  (66-1  data)  with 
the  Equipment  Status  Report  Numbers  (65-110  data)  in  only  54%  of  the  total 


reported  malfunctions.  In  addition,  some  of  the  malfunctions  involved 


multiple  board  or  part  replacements  without  any  indication  of  which  board 


or  part  was  the  primary  cause  of  failure.  In  other  instances,  a single 


board  replacement  was  indicated  in  the  data  without  any  follow-on  part 
failures  or  replacements.  It  was  also  found  that  some  of  the  failures  were 
in  redundant  channels  and  did  not  cause  system  outages  or  were  in  associated 
test  equipment.  Other  malfunctions  listed  in  the  data  were  discovered 
during  the  performance  of  daily  or  phased  inspection  for  which  the  equipment 
is  taken  down  on  a scheduled  basis. 

Lastly,  some  apparent  malfunctions  were  listed  with  extensive 
troubleshooting  times,  but  with  no  subsequent  repair  or  replacement  actions. 
It  was  thus  necessary  to  establish  a method  for  classifying  the  malfunctions, 
as  the  data  were  being  compiled,  so  that  a count  of  failures  could  be 
reasonably  made.  The  following  general  categories  were  established  for 
this  purpose: 


TABLE  1.  FIELD  MALFUNCTION  CLASSIFICATIONS 


CATEGORY 

DESCRIPTION 

0 

Multipit  board  replacement/equipment  level  failure. 

1 

Single  board  replacement  with  part  level  data. 

2 

Single  board  replacement  without  part  level  data. 

3 

Adjustments  (action  taken  codes  "L"). 

4 

Replacement  of  "minor"  parts  (action  taken  code  "G"). 

S 

Equipment  malfunction  listed  only  In  AFRM  65-110, 
parts  level  data  correlated  with  AFM  66-1  data. 

6 

Troubleshooting  (action  taken  code  "Y"),  with  no 
replacement  actions  listed. 

7 

Cleaning  (action  taken  code  "V"). 

and: 

A 

Caused  downtime  (unscheduled  maintenance). 

B 

Downtime  (scheduled  maintenance). 

C 

Failure  In  redundant  channels  and/or  did  not  cause 
system  downtime. 

0 

Failure  of  off-line  test  equipment. 

E 

Failure  listed  In  AFM  66-1,  but  not  In  AFM  65-110  data. 

The  Equipment  R&M  Logs  were  initially  compiled  using  one  year  of 


field  operational  data  on  31  equipments  which  were  located  at  different 
Air  Force  sites  within  the  United  States.  As  shown  in  Figure  2,  the  data 
were  used  in  a preliminary  analyses  contained  in  Section  3.4  of  the  report, 
to  determine  if  the  data  were  of  sufficiently  good  quality  to  attempt 
performing  further  analyses.  The  preliminary  analysis  indicated  that  the 
data  were  reasonably  usable  for  more  detailed  part  level  and  failure/repair- 
time distribution  analyses.  Comparisons  of  predicted  versus  observed  failure 
rates  are  made  in  Section  4 of  the  report  for  the  parts  and  circuit  boards 
contained  in  the  equipment.  An  additional  nine  months  of  field  operation 
data  were  compiled  for  25  of  the  31  equipments,  which  were  combined  with  the 
data  collected  earlier  for  the  preliminary  analyses.  Using  the  additional 
data,  failure  distribution  analyses,  including  trend  tests  and  goodness-of- 
fit  tests,  are  performed  in  Section  5 and  comparisons  with  prediction  and 
demonstration  statistical  distribution  assumptions  are  made.  The  field 
maintainability  characteristics  of  the  equipment  are  evaluated  in  Section  6 
of  the  report.  Conclusions  and  recomnendations  are  presented  in  Section  7. 

3.3  Failure/Malfunction  Classifications 


In  the  process  of  compiling  the  Equipment  R&M  Logs,  it  was  found 
that  both  the  AFM  66-1  and  AFM  65-110  data  records  were  incomplete.  For 
example,  it  was  possible  to  match  the  Job  Control  Number  (66-1  data)  with 
the  Equipment  Status  Report  Numbers  (65-110  data)  in  only  54%  of  the  total 

i 

reported  malfunctions.  In  addition,  some  of  the  malfunctions  involved 
multiple  board  or  part  replacements  without  any  indication  of  which  board 
I or  part  was  the  primary  cause  of  failure.  In  other  instances,  a single 


board  replacement  was  indicated  in  the  data  without  any  follow-on  part 
failures  or  replacements.  It  was  also  found  that  some  of  the  failures  were 
in  redundant  channels  and  did  not  cause  system  outages  or  were  in  associated 
test  equipment.  Other  malfunctions  listed  in  the  data  were  discovered 
during  the  performance  of  daily  or  phased  inspections  for  which  the  equipment 
is  taken  down  on  a scheduled  basis. 

Lastly,  some  apparent  malfunctions  were  listed  with  extensive 
troubleshooting  times,  but  with  no  subsequent  repair  or  replacement  actions. 
It  was  thus  necessary  to  establish  a method  for  classifying  the  malfunctions, 
as  the  data  were  being  compiled,  so  that  a count  of  failures  could  be 
reasonably  made.  The  following  general  categories  were  established  for 
this  purpose: 


5 


TABLE  1.  FIELD  MALFUNCTION  CLASSIFICATIONS 


CATEGORY DESCRIPTION 

0 Multiple  board  replacement/equipment  level  failure. 

1 Single  board  replacement  with  part  level  data. 

2 Single  board  replacement  without  part  level  data. 

3 Adjustments  (action  taken  codes  "L"). 

4 Replacement  of  "minor"  parts  (action  taken  code  "G"). 

5 Equipment  malfunction  listed  only  In  AFRM  65-110, 

parts  level  data  correlated  with  AFM  66-1  data. 

6 Troubleshooting  (action  taken  code  "Y"),  with  no 

replacement  actions  listed. 

7 Cleaning  (action  taken  code  "V"). 


Caused  downtime  (unscheduled  maintenance). 

Downtime  (scheduled  maintenance). 

Failure  in  redundant  channels  and/or  did  not  cause 
system  downtime. 

Failure  of  off-line  test  equipment. 

Failure  listed  In  AFM  66-1,  but  not  In  AFM  65-110  data 


r 


The  Equipment  R&M  Logs  were  initially  compiled  using  one  year  of 
field  operational  data  on  31  equipments  which  were  located  at  different 
Air  Force  sites  within  the  United  States.  As  shown  in  Figure  2,  the  data 
were  used  in  a preliminary  analyses  contained  in  Section  3.4  of  the  report, 
to  determine  if  the  data  were  of  sufficiently  good  quality  to  attempt 
performing  farther  analyses.  The  preliminary  analysis  indicated  that  the 
data  were  reasonably  usable  for  more  detailed  part  level  and  failure/repair- 
time distribution  analyses.  Comparisons  of  predicted  versus  observed  failure 
rates  are  made  in  Section  4 of  the  report  for  the  parts  and  circuit  boards 
contained  in  the  equipment.  An  additional  nine  months  of  field  operation 
data  were  compiled  for  25  of  the  31  equipments,  which  were  combined  with  the 
data  collected  earlier  for  the  preliminary  analyses.  Using  the  additional 
data,  failure  distribution  analyses,  including  trend  tests  and  goodness-of- 
fit  tests,  are  performed  in  Section  5 and  comparisons  with  prediction  and 
demonstration  statistical  distribution  assumptions  are  made.  The  field 
maintainability  characteristics  of  the  equipment  are  evaluated  in  Section  6 
of  the  report.  Conclusions  and  recoimendations  are  presented  in  Section  7. 

3. 3 Fa i lure/Mai  function  Classifications 

In  the  process  of  compiling  the  Equipment  R&M  Logs,  it  was  found 
that  both  the  AFM  66-1  and  AFM  65-110  data  records  were  incomplete.  For 
example,  it  was  possible  to  match  the  Job  Control  Number  (66-1  data)  with 
the  Equipment  Status  Report  Numbers  (65-110  data)  in  only  54%  of  the  total 
reported  malfunctions.  In  addition,  some  of  the  malfunctions  involved 
multiple  board  or  part  replacements  without  any  indication  of  which  board 
or  part  was  the  primary  cause  of  failure.  In  other  instances,  a single 


board  replacement  was  indicated  in  the  data  without  any  follow-on  part 
failures  or  replacements.  It  was  also  found  that  some  of  the  failures  were 
in  redundant  channels  and  did  not  cause  system  outages  or  were  in  associated 
test  equipment.  Other  malfunctions  listed  in  the  data  were  discovered 
during  the  performance  of  daily  or  phased  inspections  for  which  the  equipment 
is  taken  down  on  a scheduled  basis. 

Lastly,  some  apparent  malfunctions  were  listed  with  extensive 
troubleshooting  times,  but  with  no  subsequent  repair  or  replacement  actions. 
It  was  thus  necessary  to  establish  a method  for  classifying  the  malfunctions, 
as  the  data  were  being  compiled,  so  that  a count  of  failures  could  be 
reasonably  made.  The  following  general  categories  were  established  for 
this  purpose: 


TABLE  1.  FIELD  MALFUNCTION  CLASSIFICATIONS 


CATEGORY 

DESCRIPTION 

0 

Multiple  board  replacement/equipment  level  failure. 

1 

Single  board  replacement  with  part  level  data. 

2 

Single  board  replacement  without  part  level  data. 

3 

Adjustments  (action  taken  codes  "L"). 

4 

Replacement  of  "minor"  parts  (action  taken  code  "G"). 

5 

Equipment  malfunction  listed  only  In  AFRM  65-110, 
parts  level  data  correlated  with  AFM  66-1  data. 

6 

Troubleshooting  (action  taken  code  "Y"),  with  no 
replacement  actions  listed. 

7 

and: 

Cleaning  (action  taken  code  "V"). 

A 

Caused  downtime  (unscheduled  maintenance). 

B 

Downtime  (scheduled  maintenance) . 

C 

Failure  in  redundant  channels  and/or  did  not  cause 
system  downtime. 

0 

Failure  of  off-line  test  equipment. 

E 

Failure  listed  in  AFM  66-1,  but  not  in  AFM  65-110  data, 

2 2 


For  example,  classification  A1  indicates  an  unscheduled  maintenance 


incident  which  caused  a total  system  outage,  where  a single  board  was  re- 
placed with  corresponding  part  level  replacement  data  reported.  Classification 
of  the  malfunctions  in  this  manner  thus  permitted  greater  flexibility  in 
the  data  analyses  and  also  enabled  an  evaluation  of  the  analysis  results 
in  terms  of  the  data  completeness. 

3.4  Preliminary  Analysis  of  the  Field  Data 

The  preliminary  analysis  in  this  section  includes  a summary  of 
the  classified  failures  for  all  31  equipments,  calculations  of  the  field 
MTBF  for  each  equipment  and  comparisons  of  the  failure  rates  of  the  pre- 
diction model  with  those  derived  from  the  field  data.  The  purpose  of  the 
preliminary  analysis  was  to  determine  if  the  data  were  of  adequately  good 
quality  to  attempt  more  detailed  evaluations.  All  of  the  calculations  of 
failure  rates  (MTBF)  in  this  section  were  based  on  the  assumption  that  the 
failure  rates  are  constant. 

3.4.1  Malfunction  Classification  Sumnary  and  Failure  Counts 

Table  2 provides  a summary  of  the  classified 
failure  data  for  the  31  equipments.  During  the  one  year  data  sample  period 
the  31  equipments  were  operated  for  a total  of  169,687  hours.  Not  all  of  the 
equipments  were  operated  for  the  full  year  since  some  had  been  placed  into 
field  operation  at  various  times  within  the  year.  The  failure  categories 
described  in  the  previous  Section  3.3  were  used  as  a basis  for  either  exclud- 


TABLE  2.  FAILURE  CLASSIFICATION  SUMMARY 
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/ 

/ 

MALFUNCTION  CATEGORY* 


0 

1 

2 

4 

5 

TOTALS 

3 

6 

7 

A 

11 

39 

42 

20 

76 

188 

71 

5 

S' 

E 

5 

35 

13 

4 

- 

57 

8 

13 

1 

TOTALS 

16 

74 

55 

24 

76 

245 

79 

18 

6 

B 

1 

11 

7 

1 

- 

20 

4 

- 

- 

C 

- 

25 

11 

21 

13 

70 

9 

3 

13 

D 

1 

13 

8 

4 

9 

35 

9 

2 

3 

TOTALS 

2 

49 

26 

26 

22 

125 

22 

5 

16 

* S«e  Table  1,  Section  3.3 


Cleaning  actions  (Categories  A7,  E7),  failures  in  off-line 
test  equipment  (Category  D)  and  troubleshooting  (Categories  A6,  E6)  were 
excluded  from  the  failure  count.  Malfunctions  which  were  in  redundant 
channels  (Category  C)  and  did  not  cause  system  outages  were  not  counted 
against  the  system  MTBF.  None  of  the  failures  in  the  redundant  height 
shelves  were  found  to  cause  the  system  to  go  down.  The  primary  failure 
count  of  unscheduled  corrective  maintenance  incidents  were  the  classifica- 
tions A0-A2,  A4,  A5  and  E0-E4,  which  resulted  in  a total  of  245  failures. 
Unscheduled  "adjustments"  (total  79,  Category  3)  and  the  maintenance  actions 
which  were  performed  as  the  result  of  preventive  maintenance  (total  20, 
Category  B)  were  grouped  and  added  separately  to  the  failure  count  to  assess 
their  relative  effects  on  the  equipment  MTBF.  The  reasons  for  considering 
these  malfunction  categories  in  the  count  of  failures  are  discussed. 

Most  of  the  maintenance  actions  which  were  reported  as 
2 4 


iiA. 


"adjustments"  involved  unscheduled  downtimes  of  very  short  duration  (1-3 
minutes).  Others,  however,  had  longer  associated  downtimes  (30-60  minutes) 
and  there  is  some  question  as  to  whether  some  should  perhaps  be  counted  as 
failures.  A case  can  be  made  for  counting  some  of  the  adjustments  as 
failures* given  that  the  data  provide  adequate  detail  regarding  the  nature 
and  reason  for  the  adjustment.  However,  it  was  not  possible  to  distinguish 
from  the  source  data,  which  "adjustments"  were  legitimately  countable* 
therefore,  for  general  comparison  purposes  all  were  grouped  and  separately 

combined  with  the  primary  failure  count. 

t 

Maintenance  actions  which  were  performed  during  preventive 

* 

(scheduled)  maintenance  periods  were  found  to  contain  part,  as  well  as 
board  replacements.  It  was  foiled,  through  a visit  to  one  of  the  field  sites 
(Saratoga  Springs  NY)  that,  on  occasion,  corrective  maintenance  is  postponed 
to  periods  of  scheduled  downtime  (normally  once  per  day).  Therefore,  rather 
than  exclude  the  preventive  maintenance  actions  from  the  failure  count, 

they  were  also  grouped  and  separately  combined  into  the  total  failure  count. 

<9 

The  average  MTBF  for  all  sy^fems  combined  using  the 

classification  (A0-A2.  A4,  A5)  and  (E0-E4)  was  calculated  as  169,687  = 693 

245 

hours  which  compares  favorably  with  the  predicted  value  of  578  hours.  When 
the  Category  "B"  (preventive  maintenance)  failures  are  included,  the  MTBF  is 
169,687  = 640  hours  and  if  the  adjustments  are  counted  the  MTBF  is 

■ m ” 

169,687  = 493  hours. 

384 


Tt  should  be  noted  that  the  standard  Air  Force  computer 


printouts  which  provide  MTBF  estimates  of  the  equipment  would  have  included 
all  Category  C (60  reported  failures  in  redundant  height  shelves)  and 
Category  D (35  reported  failures  in  off-line  test  equipment)  as  well  as  the 
79  "adjustments"  (Category  3)  in  the  MTBF  calculation,  thus  yielding  a MTBF 
of  396  hours. 


The  need  for  consistent  field  failure  definition  becomes 
evident  from  the  former  discussions.  Data  from  the  field  should  be  suf- 
ficiently detailed  so  that  bonafide  equipment  failures  can  be  distinguished 
from  non- failures.  The  reliability  model  should  be  available  for  the  analysis 
so  that  proper  accounting  of  failures  in  redundant  channels  can  be  made.  In 
addition,  a review  of  the  equipment  configuration  should  be  made  to  ensure 
that  the  same  equipment  components,  assemblies,  etc.,  considered  in  the 
design  prediction  are  also  included  in  the  field  reliability  evaluation. 

As  was  found  with  the  AN/FYQ-47,  the  off-line  test  equipment  was  included 
as  part  of  the  AN/FYQ-47  configuration  in  the  work  unit  code  manuals.  The 
standardized  computer  procedures,  therefore,  count  the  off-line  test  equip- 
ment failures  (Category  D)  against  the  AN/FYQ-47. 

3.4.2  Equipment  MTBF  by  Serial  Number 

Table  3 provides  a summary  of  the  operating  time, 
number  of  failures  (unscheduled,  scheduled  and  adjustment  maintenance 
actions  counted  separately)  and  respective  MTBFs  for  each  of  the  31  equip- 
ments located  at  different  field  sites.  Generally,  there  is  a rather  large 
variability  in  the  MTBFs  among  the  various  sites.  Several  explanations 
for  the  variability  can  be  posed.  First,  omissions  in  the  reporting  of 


malfunctions  could  have  occurred  at  some  of  the  sites.  For  example,  the 
omission  of  one  or  two  failures  at  those  sites  which  had  a low  number  of 
reported  failures  (i.e.,  < 4)  could  result  in  a significant  change  in  the 
MTBF  for  the  equipment  located  at  a given  site,  yet  not  affect  the  overall 
MTBF  average  significantly.  Secondly,  for  many  of  the  reported  maintenance 
actions,  complete  traceability  and  verification  of  the  malfunction  was 
not  possible. 


TABLE  3.  EQUIPMENT  MTBF  BY  SERIAL  NUMBER 
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141 

2844.8 

4 

- 

1 

711.2 

570.0 

95 

1976.3 

10 

1 

1 

197.6 

179.7 

164.7 

93 

1987.3 

7 

- 

1 

283.9 

248.4 

149 

4952.5 

3 

- 

1 

1650.8 

— 

1238.1 

TOTALS 

169.687.3 

245 

20 

79 

692.6 

640.3 

493.0 

1 


Thus,  so-called  failures  could  have  been  malfunctions  for  which  subsequent 
off-line  maintenance  might  have  indicated  "no  trouble  found".  Third, 
initially  high  failure  rates  could  be  present  in  some  of  the  fielded  equip- 
ment because  of  an  inadequate  weeding-out  of  early  failures  (e.g.,  poor 
workmanship,  substandard  parts  or  materials)  during  the  manufacturing 
process.  The  analysis  in  Section  5 of  the  report  is  directed  toward  examin- 
ing the  failure  distribution  characteristics  of  the  AN/FYQ-47  in  the  field. 

3.4.3  Reliability  Prediction  Model  and  Field  Failure  Rate 

Comparisons. 

Figure  4 below,  provides  a comparison  of  the  predicted  and 
observed  failure  rates  using  the  AN/FYQ-47  reliability  model  as  the  basis 


for  comparison. 
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FIGURE  4. 


As  can  be  noted,  the  comparisons  are  remarkably  good,  in  spite  of  the 
limitations  presented  by  the  data.  Included  in  the  preparation  of  Figure  4 
were  all  the  unscheduled  maintenance  incidents  which  caused  system  downtime 
(Categories  A0-A5,  A0-A2,  A4,  A5,  E0-E4)  of  Table  1.  For  the  redundant 
height  shelves  the  failure  rate  comparisons  were  made  on  a simplex  basis. 
None  of  the  height  shelf  failures,  as  indicated  by  the  AFM  65-110  data,  re- 
sulted in  system  outage.  Of  the  245  unscheduled  maintenance  incidents,  it 
was  not  possible  to  isolate  22  to  any  of  the  subsystem  groups  within  the 
reliability  model . 

The  results  of  the  former  comparisons  were  encouraging  in  that 
they  indicate  that  the  malfunction  reporting  from  the  field,  at  least  to  the 
subsystem  level,  was  reasonably  complete  and  accurate.  Since  obtaining  such 
correlations  by  chance  alone  is  fairly  remote,  it  was  decided  to  continue 
with  more  detailed  reliability  evaluations  at  the  part  and  circuit  board 
levels.  These  analyses  and  results  are  discussed  in  the  next  Section  4. 

3. 5 Summary  of  Findings 

(a)  The  data  are  generally  usable  for  more  detailed  analyses  at 
the  part  level  and  for  failure  distribution  analyses  at  the  equipment  level. 

(b)  By  combining  chronologically  processed  data  from  the  AFM  66-1 
and  AFM  65-110  data  systems,  R&M  data  elements,  such  as  the  time- to- failure, 
and  the  repair  times  needed  for  R&M  assessments  can  be  obtained. 

(c)  Use  of  supplemental  information  provided  by  the  equipment 
technical  manuals,  illustrated  parts  breakdown  and  work  unit  code  manuals 


permits  obtaining  additional  needed  data  elements,  as  well  as  resolution 
of  discrepancies  in  the  field  data. 
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(d)  Estimates  of  MTBF  provided  in  standard  Air  Force  data  products 
cannot  be  relied  upon  for  use  in  making  comparisons  against  specified  or 
predicted  values.  This  is  due  primarily  to  differences  in  failure  definitions. 
Equipment  configuration  misidentifications  between  the  design  and  field  also 
contribute  to  the  error  in  MTBF  estimates  provided  by  the  standard  data 
system  outputs. 

(e)  Some  bonafide  equipment  failures  may  be  reported  as  "adjust- 
ments", or  may  be  reported  as  preventive  maintenance,  depending  on  site 
operating  procedures. 

(f)  A substantial  percentage  of  the  data  are  incomplete  at  the 
part  level  which  obviously  affects  the  usability  of  the  data  for  performing 
more  detailed  analyses. 

(g)  For  equipments  operating  in  environments  similar  to  the 
AN/FYQ-47,  it  is  possible  to  obtain  operating  time  as  well  as  time-between- 
failure  data  at  the  system  level. 

(h)  It  is  not  possible  to  obtain  more  detailed  breakdowns  of 
corrective  maintenance  times  such  as  fault  location  time,  fault  correction 
time,  etc.  However,  provisions  for  reporting  delay  times  and  downtimes  for 
maintenance  are  included  in  the  data  systems  procedures. 

(i)  There  are  inconsistencies  and  omissions  in  the  data  reported 
from  different  sites  and  in  separate  data  systems.  Much  improvement  can 
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obviously  be  accomplished  by  providing  unambiguous  instructions  on  how  the 
data  collection  forms  are  to  be  filled  out  for  reporting  failures.  Such 
instructions  should  be  unique,  where  possible,  to  the  system  for  which  the 
data  are  being  collected. 

(j)  Obtaining  matching  part/assembly  identifications  between  those 
used  for  the  prediction  and  those  obtained  from  the  field  failure  data  is  one 
of  the  most  time-consuming  tasks  in  attempting  to  perform  field  R&M  analyses. 
Use  of  part  vendor  assigned  generic  identifications,  equipment  contractor 
assigned  part  numbers,  and  government  federal  stock  codes,  work  unit  codes, 
etc.,  contribute  to  such  difficulties. 

(k)  Chronological  sorting  of  the  data  enables  obtaining  operating 
time  between  failure  information  as  well  as  facilitating  the  analyses.  Machine 
processing  of  the  data  in  the  Equipment  R&M  Log  format  would  be  very  useful 

for  R&M  analysis  purposes  provided  that  the  data  are  complete  and  properly 
edited  prior  to  processing. 

(l)  The  reliability  model  for  the  equipment  should  be  available 
for  the  analysis  so  that  proper  accounting  of  failures  in  redundant  channels 
can  be  made. 

(m)  Review  of  the  equipment  configuration  should  be  made  to 
ensure  that  the  same  major  equipment  components  are  included  in  the  field 
R&M  evaluations  as  were  used  in  the  reliability  prediction  and  demonstration. 


: 


4.0  PARTS  AND  CIRCUIT  BOARD  DATA- PREDICTED/ FI  ELD- OBSERVED  COMPARISONS 


Appendix  C contains  tables  identifying  the  parts  and  circuit  boards  used 
in  the  AN/FYQ-47,  their  respective  quantities,  predicted  failure  rates, 
expected  field  failures  and  observed  field  replacement  rates.  It  is  again 
noted  that  some  of  the  maintenance  actions  involved  multiple  part/board 
replacements  without  the  primary  cause  of  failure  being  identified  in  the 
data.  Therefore,  all  replacements  were  included  in  Column  4 of  the  tables 
of  Appendix  C and  replacement  rates  rather  than  failure  rates  were  calculated 
in  Column  5.  Calculations  in  the  Appendix  C tables  were  based  upon  the  total 
accumulated  operating  time  of  169,687  hours  for  all  31  systems. 

4.1  Parts  Data 


Table  4 below  provides  a summary  of  the  Appendix  C part  data  showing 
part  counts,  expected  failures  and  the  observed  field  replacements  for  the 
various  electronic  part  types  used  in  the  AN/FYQ-47. 


TABLE  4.  EXPECTED  FAILURES  AND  FIELD  REPLACEMENTS 
FOR  AN/FYQ-47  ELECTRONIC  PARTS 


PART  COUNT 

EXPECTED  FAILURES 

FIELD  REPLACEMENTS 

Monolithic 

Microcircuits 

6250 

75.7 

140 

Diodes 

647 

2.2 

3 

Transistors 

1689 

b 

31.4 

84 

Capacitors 

4478 

12.9 

1 

Resistors 

3968 

51.9 

5 

TOTALS 

174.1 

233 

actual  quantity  1832 

Data  was  not  available  on  4 transistor  types 


Although  the  overall  total  field  replacements  shown  in  Table  4 are  greater 
(by  about  33%)  than  what  the  prediction  data  might  have  indicated,  there 
are  even  larger  and  inconsistent  differences  within  the  various  part  types. 
Since  comparisons  of  total  replacements  rather  than  actual  failures  were 
made,  one  might  expect  to  see  the  total  replacements  always  larger  than  the 
total  expected  failures.  It  must  also  be  noted,  however,  that  some  of  the 
data  were  incomplete  with  regard  to  the  reporting  of  part  level  replacements 
(e.g..  Failure  Classes  A2,  E2,  AO,  EO,  A5  in  Section  3.3).  If  each  of  the 
former  maintenance  actions  had  resulted  in  part  replacement,  then  obviously 
the  differences  in  the  totals  would  have  been  even  larger.  The  data  of 
Table  4 would  initially  indicate  rather  poor  correlations,  as  well  as  cast 
serious  doubts  as  to  the  usefulness  of  the  field  data  for  any  analyses  to 
the  part  level.  However,  when  the  data  for  each  of  the  part  types  are 
examined  further,  some  interesting  findings  result. 

4.1.1  Microcircuits 

The  6250  microcircuits  contained  in  the  AN/FYQ-47  were 
exposed  to  over  one  billion  (1060.54  x 106)  operating  hours  during  the  time 
period  in  which  the  data  were  collected.  Figure  5 is  a plot  of  the  expected 
number  of  failures  versus  the  number  of  replacements  for  each  of  the  21 
microcircuit  device  types  used  in  the  equipment.  As  shown  in  this 
figure,  the  correlation  between  the  expected  failures  and  field  observed 
replacements  is  remarkably  good  for  the  majority  of  the  devices.  These 
results  suggest  that,  at  least  for  the  microcircuit  devices,  fairly  consistent 
and  accurate  reporting  of  field  replacements  were  made  by  the  field 


technicians  at  alj_  the  sites.  The  digital  nature  of  the  devices  may  also 
have  a bearing  on  the  good  correlations  because  the  failures  generally  can 
be  more  easily  and  positively  identified. 


6250  DEVICES 

1060.5  x 106  DEVICE  HOURS 
21  DEVICE  TYPES 
TOTAL  REPLACEMENTS  140 
EXPECTED  FAILURES  75.7 


(A)  - QUAD  2 - INPUT  HAND/ NOR 

PART  #1700-2510 

(B)  - J-K  FLIP-FLOP  AND/OR  INPUT 

PART  #1700-2676 


0 2 4 6 8 10  12  14  16  18  20 

EXPECTED  FAILURES 


FIGURE  5.  EXPECTED  FAILURES  VS  FIELD  REPLACEMENTS 
FOR  AN/FYQ-47  MONOLITHIC  MICROCIRCUITS 


The  devices  labelled  A and  B in  Figure  5 are  the  Quad  2-Input  Nand/ 
Nor  (Part  #1700-2510)  and  the  J-K  Flip-Flop  and/or  Input  (Part  #1700-2676) 
respectively,  which  are  listed  in  Appendix  C.  These  devices  display 


a considerable  difference  from  predicted  values  and  represent  21%  of  the 
total  microcircuit  population  in  the  equipment.  Samples  of  these  failed 
devices  would  obviously  be  good  candidates  for  detailed  failure  analysis, 
to  determine  failure  causes  and  corrective  actions.  Note  that  the  pre- 
dictions provide  a very  useful  tool,  since  they  serve  as  a baseline  against 
which  field  failure  frequency  can  be  compared  and  potential  reliability 
problem  areas  isolated.  It  should  also  be  noted,  that  use  of  the 
raw  field  data,  as  it  is  currently  being  generated,  for  verifying  or  modify- 
ing handbook  (i.e.,  MIL-HDBK-217B)  predicted  failure  rate  values  (See 
Objective  "f"  in  Section  1)  would  be  a highly  questionable  practice.  Additional 
failure  circumstance  data,  as  well  as  detailed  laboratory  analyses  of  failed 
parts,  are  required  in  order  to  meaningfully  achieve  such  analyses  objectives. 
Data  analyses,  using  the  prediction  data  as  a comparison  base,  does,  however, 
provide  a means  for  selecting  field  failed  devices  which  offer  the  greatest 
pay-off  in  terms  of  laboratory  failure  analysis  resources  to  be  expended 
versus  tne  analyses  objectives  which  are  desired. 

4.1.2  Diodes  a:,d  Transistors 

During  the  period  over  which  the  data  were  collected,  the 
diodes  had  slightly  over  100  million  device  hours,  109.785  x 10^,  and  the 
transistors  286.601  x 10&  device  hours.  Overall,  the  diodes  of  Table  4 
show  good  correlation  (2.2  vs  3)  between  the  expected  and  observed  replace- 
ment rates.  However,  even  with  the  large  exposure  hours  available,  individual 
diode  types  had  limited  exposure  time  due  to  their  relatively  small  quantities. 
Therefore,  very  little  can  be  said  about  the  field  performance  of  the  diode 


population  in  the  equipment. 


The  transistors,  on  the  other  hand,  show  a large  disparity  between 
the  expected  and  observed  replacement  rates.  Figure  6 rather  vividly 
illustrates  the  poor  correlations. 


FIGURE  6.  EXPECTED  FAILURES  VS  FIELD  REPLACEMENTS 
FOR  AN/FYQ-47  TRANSISTORS 


Although  the  data  are  still  somewhat  limited,  three  possible  reasons 


for  the  poor  correlations  can  be  posed: 


(a)  The  values  of  the  predicted  failure  rates  for  most  of 


the  transistors  were  overly  optimistic. 


(b)  The  transistors  are  generally  of  poor  quality  due  to 


inadequate  part  screening  and  controls. 


(c)  Transistors  were  replaced  in  the  field  by  the  technicians 
which  were  not  the  primary  cause  of  system  failures  (i.e.,  either  the 
transistors  upon  retest  would  test  "good"  or  they  represented  secondary 
failures). 
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It  is  not  possible  to  isolate  the  cause  of  the  poor  correlations  to 
any  one  of  the  above  reasons,  given  the  quality  of  the  current  field  data. 
However,  if  the  data  were  at  least  sufficiently  accurate  to  rule  out  entirely 
reason  (c),  then  additional  efforts  (i.e.,  laboratory  analyses  of  selected 
failed  parts)  could  be  concentrated  to  determine  the  cause  of  the  poor 
correlations  and  determine  corrective  actions. 


The  data,  however,  are  still  useful  for  isolating  potential  re- 
liability problem  areas.  For  example,  the  devices  labelled  A&B  in  Figure  6 
have  the  highest  overall  replacement  rates.  When  the  data  for  these  devices 
(see  Figure  6 and  Appendix  C)  are  compared  as  shown  in  Table  5 below. 


it  becomes  quite  obvious  that  device  A,  at  least  until  additional  failure 
circumstance  data  are  reviewed  and  perhaps  more  detailed  failure  analysis 
is  done,  represents  a reliability  problem  area. 

TABLE  5.  TRANSISTOR  REPLACEMENT  COMPARISONS 


It  should  again  be  emphasized  that  if  there  was  positive 
identification  and  verification  of  the  primary  cause  of  field  failures  to 
the  part  level,  without  omissions,  then  part  level  analyses  of  this  type 
could  be  extremely  more  valuable  for  achieving  field  reliability  assessment 
objectives. 

4.1.3  Capacitors  and  Resistors 

The  capacitors  and  resistors  in  the  equipments  sampled  had 
respectively  759.86  x 106  and  673.32  x 10^  device  hours  during  the  data 
collection  period.  The  data  of  Table  4 show  the  largest  difference  between 
the  predicted  and  field  replacement  rates  for  these  parts,  but  in  a direction 
different  from  the  previously  listed  part  types  (i.e.,  expected  failures  « 
field  replacements).  If  one  were  to  assume  that  the  field  replacement  rates 
are  reasonably  correct  (i.e.,  few  omissions  occurred)  then  some  of  the 
failure  rates  used  in  the  predictions  were  obviously  pessimistic.  For 
example,  the  RL  (MIL-R-22684)  fixed  film  resistors  shown  in  Appendix  C, 
represent  85%  of  the  total  resistor  population  in  the  equipment  and  account 
for  the  majority  of  the  disparity  between  predicted  and  observed  replacement 
rates.  The  predicted  failure  rate  used  for  the  fixed  film  resistors  was 
.072  x 10"6,  which  differs  by  at  least  an  order  of  magnitude  from  the 
observed  replacement  rate  of  .004  x 10" 6.  On  the  other  hand,  there  may 
have  been  a tendency  for  the  technicians  in  the  field  to  consider  capacitors 
and  resistors  "minor"  parts  and  therefore,  not  record  their  replacement. 

The  following  questions  can  therefore  be  posed:  Are  the  failure  rate 
predictions  for  the  capacitors  and  resistors  overly  pessimistic?  Was 
there  widespread  omission  at  al 1 the  sites  in  reporting  capacitor  and 


resistor  replacements?  These  questions,  of  course,  cannot  be  reasonably 
answered  using  the  existing  data.  However,  if  data  collection  procedures 
were  developed  to  ensure  complete  and  accurate  recording  of  failure  data 
to  the  part  level,  then  the  data  could  be  very  useful  for  evaluations  of 
the  type  discussed  above  as  well  as  for  other  R&M  assessment  objectives. 

4.2  Circuit  Board  Data 

Table  6 below  shows  the  respective  quantities,  expected  failures 
and  field  replacements  for  the  logic  and  hybrid  circuit  boards  contained  in 
the  equipment.  The  logic  boards  are  comprised  primarily  of  microcircuits, 
while  the  hybrid  boards  contain  mostly  discrete  components  (semiconductors, 
capacitors,  resistors)  as  well  as  some  microcircuits.  The  same  operating 
time  base  used  for  the  parts  in  the  previous  section  was  used  for  the 
expected  failure  calculations  of  the  circuit  boards. 


TABLE  6.  EXPECTED  FAILURES  AND  FIELD  REPLACEMENTS 
FOR  AN/FYQ-47  CIRCUIT  BOARDS 


TYPE 

QUANTITY 

EXPECTED  FAILURES 

FIELD  REPLACEMENTS 

Logic 

617 

128.9 

160 

Hybrid 

1 5 1 c 

67.2 

52 

TOTALS 

768 

196.1 

212 

c actual  quantity  191 
comparable  data  was  not  available  on  2 
board  types 


The  totals  of  Table  6 compare  rather  well,  but  again  a reversal 
in  the  number  of  field  replacements,  from  greater  than  to  less  than  the 


expected  failures,  is  evident  in  comparing  the  logic  and  hybrid  boards. 


Since  the  hybrid  boards  contain  a majority  of  discrete  passive  components. 


these  results  as  expected,  would  tend  to  agree  with  the  parts  data  (capacitors 


and  resistors)  of  the  previous  section  4.1.3. 


4.2.1  Logic  Boards 


Figure  7 is  a plot  of  the  expected  failures  versus  field 


replacements  for  the  617  logic  boards  contained  in  each  of  the  31  equipments. 


FIGURE  7.  EXPECTED  FAILURES  VS  FIELD  REPLACEMENTS 
FOR  AN/FYQ-47  LOGIC  BOARDS 
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A total  operating  time  of  104.697  x 10^  hours  was  logged  for  the  logic 
boards  during  the  data  collection  period. 

1 

Very  good  correlation  is  evident  from  the  plotted  data. 

The  point  labelled  "A"  in  the  figure  is  the  Inverter  Board  (#1700-2171). 

The  large  number  of  replacements  for  these  boards  is  traceable  partially  to 
the  QUAD  2 INPUT  NAND/NOR  (#1700-2510)  microcircuit  (point  labelled  "A"  in 
Figure  5).  Although  the  part  is  used  extensively  throughout  the  equipment, 
the  problem  appears  at  least  to  be  localized  to  the  Inverter  Board.  Again, 
this  illustrates  how  the  field  data  can  be  used  in  conjunction  with  the 
predictions,  to  isolate  potential  reliability  problem  areas. 

The  logic  board  data  of  this  section  can  be  used  to 
illustrate  two  general  observations  regarding  the  use  of  the  Air  Force 
maintenance  data  for  reliability  assessment  purposes.  First,  isolation  of 
potential  reliability  problem  areas  can  often  only  be  realistically  done 
through  use  of  the  large  amounts  of  data  which  are  generated  through  field 
operation.  Although  no  single  equipment  is  experiencing  an  inordinately 
large  number  of  Inverter  Board  failures,  the  field  data  (for  31  equipments) 

* 

indicate  a potential  reliability  problem.  Secondly,  in  some  field  data 
analysis  computer  programs  a straight  forward  ranking  by  replacement  rate 
is  used  to  isolate  "high-burners"  or  reliability  problem  areas.  Such 
rankings,  without  comparisons  using  the  predicted  failure  rates,  can 
obviously  mask  important  information  and  lead  to  the  wrong  emphasis  being 
placed  on  items  which  intrinsically  have  a higher  failure  rate  relative  to 
other  items. 

ill 

i 


i 

x.  ^ 


«'”*  4.2.2  Hybrid  Boards 

Figure  8 is  a plot  of  the  expected  failures  versus  field 
replacements  for  151  hybrid  boards  used  in  the  equipment.  A total  of 
24.62  x 106  board  hours  were  logged  for  the  hybrid  boards  during  the  data 
collection  period. 


FIGURE  8.  EXPECTED  FAILURES  VS  FIELD  REPLACEMENTS 
FOR  AN/FYQ-47  HYBRID  BOARDS 


The  poor  correlation  which  is  evident  here  is,  of  course,  directly  related  to 
the  poor  correlation  which  was  observed  with  the  discrete  components 
(transistors,  capacitors,  resistors).  The  point  labelled  (A)  in  Figure  8 


is  the  Height  Amplifier  Board  (#1700-6099).  The  relatively  large  number  of 
replacements  for  this  board  is  traceable  to  the  2N2219A  transistor  (Table  5 
in  Section  4.1.2). 
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If  one  compares  Figure  7 containing  the  logic  board  data 
and  Figure  8 containing  the  hybrid  board  data,  little  need  is  seen  for 
obtaining  additional  information  on  the  majority  of  logic  boards,  since  the 
correlations  are  relatively  good.  However,  additional  failure  circumstance 
and  cause  data  are  necessary  to  explain  the  poor  correlations  which  are 
evident  for  the  hybrid  boards.  One  must  first  be  fairly  certain  that  the 
data  are  reasonably  complete,  that  is,  no  omissions  in  reporting  were  made. 
Secondly,  root  cause  of  failure  information  is  required,  so  that  proper 
responsibility  for  the  failures  can  be  assigned  (e.g.,  design,  substandard 
parts,  manufacturing  processes,  prediction,  overstress,  etc.)  and  corrective 
actions  determined  where  required. 


Summary  of  Findings 


Ii'  evaluating  the  findings  of  the  former  sections  (4.4.2)  the  R&M 
analysis  objectives  listed  in  Section  1 and  the  quality  of  the  part  level 
data  which  are  available  from  the  field  must  be  considered.  Obviously, 
primary  failure  cause  information  identified  to  the  part  level  is  essential 
if  the  objectives  are  to  be  meaningfully  achieved.  In  fact,  for  most  of 
the  analysis  objectives  listed  in  Section  1 to  be  fully  achievable,  detailed 
failure  cause  information  resulting  from  laboratory  failure  analyses  of 
selected  failed  parts  is  also  required.  If,  in  making  gross  comparisons  of 
reliability  program  derived  failure  rates  with  field  estimates,  there  is 


1 


4 3 


generally  good  agreement,  then  obtaining  detailed  cause  information  is  not 
essential  to  the  analyses.  However,  when  there  are  major  inconsistencies 
in  the  estimates  either  at  the  part,  board  or  equipment  levels,  then  it  is 
essential  that  root  cause  information  be  obtained.  Without  such  data,  the 
analysis  results  can  only  be  inconclusive,  since  the  underlying  data  base  is 
incomplete. 

The  data  system  conditions  which  are  currently  hindrances  to 
attaining  most  of  the  reliability  analysis  objectives  of  Section  1 are: 

(a)  There  are  omissions  in  the  reporting  of  failed  part 
replacements  in  the  field. 

(b)  Multiple  part  replacements  are  reported  without  any 
indication  of  the  primary  cause  of  the  failure. 

(c)  Verification  of  the  failure  circumstance  data  is  not 
normally  performed  (i.e.,  that  a reported  replaced  part  is  truly  a bonafide 
failure  which  caused  the  equipment  to  fail). 

(d)  Selective  laboratory  failure  analysis  of  failed  parts 
is  not  routinely  performed  in  order  to  determine  root  cause  of  failure. 

The  Air  Force  maintenance  data  collection  systems  cannot  be 
faulted  for  conditions  (c)  and  (d).  These  conditions  must  be  rectified 
through  procedures  established  by  the  agency  responsible  for  the  R&M  data 
collection  and  analyses.  Conditions  (a)  and  (b),  however,  can  be  rectified 
by  implementing  special  data  collection  and  handling  procedures  within  the 
framework  of  current  Air  Force  data  systems.  Further  discussion  of  this 


area  will  be  provided  in  the  final  recommendations  which  are  made  in 
Section  7.2  of  the  report. 


Some  additional  findings  of  this  section  are: 

(a)  Air  Force  maintenance  data  can  be  used  to  isolate 
potential  reliability  problem  areas  (Objective  (b),  Section  1).  However,  to 
further  define  the  nature  of  the  problem,  additional  information  is  required 
which  must  be  developed  by  the  group  or  agency  performing  the  investigation. 

(b)  Data  system  outputs,  which  provide  rankings  by  hardware 
replacement  rates,  cannot  be  relied  upon  to  yield  valid  priority  information 
on  problem  areas  either  in  terms  of  cost  or  reliability.  The  reliability 
prediction  should  be  used  in  conjunction  with  such  rankings. 

(c)  Part  level  failure  rate  data  derived  from  Air  Force 
maintenance  data  systems  cannot  be  used  to  verify  or  modify  MIL-HDBKt217 
predicted  failure  rate  values  (Objective  (f).  Section  1).  This  data  must  be 
supplemented  by  the  group  performing  the  analysis  in  order  to  be  fully  usable. 

(d)  Reliability  predictions  provide  a baseline  against 
which  field  failure  rates  can  be  compared.  Use  of  the  predictions,  as  a 
reference,  enables  selection  of  failed  devices  for  analyses  which  offer  the 
greatest  return  on  resources  expended  on  laboratory  failure  analyses. 
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5.0  EQUIPMENI  FIELD  FAILURE  DISTRIBUTION  ANALYSES  • i 

In  order  to  perform  the  analyses  of  the  equipment  failure  distribution, 
it  was  necessary  to  obtain  additional  field  data.  Since  the  data  for  most 
of  the  equipments  were  rather  limited  in  terms  of  the  number  of  failures 
observed,  an  additional  nine  months  of  data  for  25  of  the  31  equipments 
were  obtained.  The  data  for  the  25  equipments  were  compiled  in  the 
equipment  R&M  Logs,  and  the  failures  were  classified  as  described  in 
Section  3.3. 

5.1  Operating  Time  Between  Failure  Data 

The  operating  times  between  failures  for  each  equipment  were  required 
to  perform  the  distribution  analyses.  Using  the  chronological  sorts  of  the 
data,  and  by  matching  either  the  ESR  Numbers  and  the  Job  Control  Numbers,  or 
the  dates  of  failure,  between  the  two  sets  of  data,  the  operating  times 
between  failures  were  obtained.  In  those  instances  where  it  was  not  possible 
to  accurately  match  the  failure  and  operating  time  data,  the  entire  equipment 
data  set  was  censored  from  any  further  analysis. 

• 

The  Equipment  R&M  Logs  provided  the  vehicle  for  compiling  the 

i 

operating  time  between  failure  data.  Further  screening  of  the  data  was  also 
performed  to  weed  out  equipment  data  sets  which  contained  questionable  data. 

For  example,  equipment  serial  number  79  (SeeTable3)  which  had  an  inordinately 
large  number  of  "adjustments"  was  censored  from  any  further  analyses.  In 
addition,  equipment  serial  numbers  160  and  123  of  Table  3,  each  with 
previously  calculated  MTBF  values  of  2602.8  hours  and  2090.6  hours 
j respectively,  were  found  to  contain  missing  data.  These  equipments,  as  well 
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as  others,  where  questionable  data  were  evident,  were  also  excluded  from 
any  further  analyses.  Small  readjustments  in  the  failure  count  were  also 
made  for  some  of  the  equipments  which  were  to  be  retained  for  the  distribution 
analyses.  It  was  found  that  the  same  failures  were  listed  in  the  two  data 
sets  under  different  coding  schemes  and  therefore,  had  been  doubly 
counted  when  the  R&M  logs  were  compiled. 

Twelve  of  the  25  equipments  were  used  for  the  distribution  analyses. 
The  additional  data  obtained  for  equipment  serial  numbers  133  and  57  were  not 
of  sufficiently  good  quality,  so  only  the  original  data  were  used  for  these 
equipments.  The  data  for  the  12  equipments  were  plotted  with  cumulative 
failures  against  cumulative  operating  time.  The  total  operating  times  only 
to  the  last  reported  failure  for  each  equipment  were  used  in  the  data  plots 
and  in  the  subsequent  analyses.  The  data  plots,  which  are  discussed  in  the 
following  section,  were  used  as  a first  step  in  developing  an  analysis 
approach. 

5.2  Plots  of  Cumulative  Field  Failures  vs  Operating  Time 

Figures  9 through  20  provide  plots  of  cumulative  failures  against 
cumulative  operating  time  for  each  of  the  12  equipments.  The  slope  of  the 
line  provided  in  each  plot,  provides  a reference  for  the  rate  of  occurrence 
of  failures  under  the  assumption  of  a constant  MTBF  equal  to  the  predicted 
value  of  579  hours.  As  can  be  noted,  some  of  the  equipments  (e.g.,  serial 
number  71)  appear  to  indicate  an  increasing  failure  frequency  (concave 
upward)  trend,  while  others  (e.g.,  serial  number  133)  seem  to  indicate 
reliability  growth  (concave  downward)  trends.  The  majority,  however, 
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FIGURES  9-12.  CUMULATIVE  FAILURES  VS  CUMULATIVE  OPERATING  TIME 
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FIGURES  17-20.  CUMULATIVE  FAILURES  VS  OPERATING  TIME 


indicate  no  clear  cut  trends  to  either  increasing  or  decreasing  failure 
frequency  with  operating  time.  In  fact,  some  of  the  equipments  show  cyclic 
patterns  over  the  operating  time  periods  observed.  These  patterns  may 
possibly  represent  just  random  fluctuations  in  the  failure  occurrences  about 
some  average  "constant"  value  or  they  may  represent  a long  term  nonconstant 
failure  rate  trend.  It  should  be  noted  that  the  assumptions  underlying  the 
reliability  prediction  model  discussed  in  Section  2.1,  would  imply  that  the 
equipment  "failure  rate"  is  constant.  Since  there  was  some  evidence  in  the 
plotted  data  that  nonconstant  trends  might  be  present,  an  analysis  approach 
was  needed  to  objectively  assess  if  such  trend  patterns  exist  in  the  failure 
data. 


5.3  Analysis  Approach 

In  many  analyses  of  repairable  system  failure  data  appearing  in 
the  literature,  the  equipment  under  study  is  assumed  to  be  as  good-as-new 
following  the  repair.  The  former  assumption  implies  that  the  reliability 
model  for  the  repaired  equipment  is  identical  to  that  of  a new  equipment 
with  operating  time  taken  at  t=0  each  time  the  equipment  is  repaired.  Stated 
differently,  the  good-as-new  assumption  implies  that  the  times  between 
failures  have  identical  and  independent  statistical  distributions.  Referring 
to  the  reliability  prediction  model  used  for  the  AN/FYQ-47,  (Section  2.1) 
for  example,  said  statistical  distribution  would  be  exponential  with  parameter 
equal  to  the  sum  of  the  failure  rates  of  the  series  elements.  Under  the 
good-as-new  assumption  the  analyses  for  repairable  system  failure  data  are 
most  often  done  by  pooling  and  ordering  the  failure  times  by  magnitude, 
i.e.,  disregarding  the  original  chronological  sequence  of  the  failures.  The 


pooled  and  ordered  data  are  then  analyzed,  usually  for  exponential ity,  using 
statistical  goodness-of-fit  tests. 

An  alternative  to  the  good-as-new  assumption  is  the  case  where, 
after  a repair,  the  equipment  is  considered  identical  to  an  equipment  which 
has  never  failed,  i.e.,  failure  times  are  taken  from  t=0  (initial  installa- 
tion) rather  than  since  the  last  repair  was  made.  This  concept  is  often 
referred  to  as  the  bad-as-old  assumption.  The  bad-as-old  assumption  implies 
that  any  parts/assemblies  replaced  during  a given  repair  have  the  same  age 
as  the  total  equipment.  Under  the  bad-as-old  assumption  the  analysis  approach 
would  be  quite  different  from  that  which  might  use  the  good-as-new  assumption. 
The  chronological  sequence  of  the  failure  would  have  to  be  retained  and  the 
analysis  model  selected  would  have  to  address  the  possibility  of  statistically 
dependent  and  nonidentically  distributed  failure  times. 

In  order  to  settle  on  an  approach  for  the  distribution  analyses, 
a quantitative  assessment  of  overall  trend  in  the  failure  data  was  needed. 

If  trends  are  found  to  be  present,  then  the  times  between  failure  should  not 
be  treated  as  identical  and  independent  observations  and  the  pooling  and 
ordering  of  the  failures  times  would  not  be  appropriate  for  the  analyses. 

On  the  other  hand,  if  no  trends  are  found,  then  pooling  and  ordering  would 
be  permissible,  and  conventional  goodness-of-fit  tests,  using  various 
distribution  assumptions,  could  be  applied  to  the  data. 

5.4  Analysis  Results 

In  the  sections  which  follow,  trend  tests  and  goodness-of-fit  tests 
are  applied  to  the  failure  and  operating  time  data  for  the  12  equipments. 


Confidence  limits  for  the  mean- time-between-fai lure  for  each  equipment  are 
also  calculated,  and  plots  of  the  fitted  vs  empirical  reliability  functions 
for  some  of  the  equipments  are  provided. 

5.4.1  Mann  Test  for  Trend 

The  Mann  Test  for  Trend  was  applied  to  the  successive 
failure  times  for  each  of  the  12  equipments  to  provide  an  objective  assess- 
ment of  increasing  or  decreasing  failure  trends  in  the  failure  data. 

Basically,  the  procedure  provides  a test  of  the  null  hypothesis  (Ho)  of  no 
trend  against  an  alternative  hypothesis  (Hi)  of  increasing  or  decreasing 
failure  frequency  with  operating  time.  If  no  trends  in  the  chronologically 
sequenced  failure  times  are  found  ( i . e . , the  null  hypothesis  cannot  be 
rejected)  then  it  can  be  concluded  that  the  good-as-new  assumption  is  reason- 
able. A description  of  the  test  procedure  and  the  results  of  the  test 
follows: 

Under  the  null  hypothesis,  there  is  no  tendency  for  success- 
ive intervals  between  failure  to  increase  or  decrease.  As  part  of  the  test, 
comparisons  between  successive  pairs  of  intervals  are  made  wherein  the  number 
of  times  an  earlier  interval  is  less  than  a later  interval  is  counted.  Given 
that  the  successive  intervals  for  each  equipment  are  denoted  by  X],  X2,  ...Xn 
then  a count  (Tn)  is  made  such  that: 

Tn  = number  of  inequalities  Xi<Xj,  i<j  + one  half  number  of 
comparisons  Xi  = Xj,  i<j 


(10)  Mann,  H.R.,  "Nonparametric  Tests  Against  Trend"  Econometrics,  1945, 
page  245-249 
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Mann  shows  that  for  n>10  the  distribution  on  Tn  is  asymtotical ly  normal 
with  parameters: 

E(Tn)  = -ili11"1.)  . 

4 


and 

a2(Tn)  = (2n , ,t,  5j(n-l)  n 
72 

where  n is  the  number  of  observations 
The  standard  normal  variable 

Zn  = ■ Tn-  +.i?---L(Tn) 

cr(  Tn ) 

can  be  used  as  the  basis  of  a test  of  significance  given  reasonably  large 
sample  sizes  for  each  equipment.  A test  of  significance  for  the  aggregate 
of  all  12  equipments  can  be  made  by  using  the  test  statistic: 

U = X(Tn  +S-  E(Tn)) 

'sjz  a2(Tn) 

Where  U is  also  a standard  normal  variable  from  a normal  population  with 
parameters  equal  to  the  sum  of  the  means  and  variances  of  Tn  for  each  of 
the  equipments. 

The  test  of  significance  is  then  given  by: 

P (U  > Ua  | Ho)  £ a where  a is  the  significance  level. 

The  results  of  the  test  are  provided  in  Table  7. 
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TABLE  7.  RESULTS  OF  MANN  TEST  FOR  TREND 


203.0 


204.3 

710.5 

204.3 

147.3 


Tn  + 1/2  - E 


-1.28 


164.5 

131.0 


115.5 

138.0 


138.0 


204.3 

147.3 

147.3 

314.3 

406.3 
406.3 


-1.35 


TOTALS 


1207.5  1139.5 


3042.5 


Using  the  data  of  Table  7,  a calculation  of: 


/\/  £ oL  (Tn) 


- = 1.34 
55.16 


for  the  test  of  significance,  indicates  that  the  hypothesis  of  no  trend  can- 
not be  rejected  at  a 5%  one-sided  level  (Ua  = 1.645)  for  the  pooled  data  set. 
Equipment  serial  numbers  133  and  93,  taken  individually,  could  be  rejected 
at  the  5%  level  and  a claim  made  that  these  equipments  are  exhibiting 


U. 


reliability  growth.  Reliability  growth  models  could  be  fitted  to  the 
chronologically  sequenced  failure  times  for  these  equipments  and  an  appropriate 
goodness-of-fit  test  applied.  These  analyses  were  not  done  as  part  of  this 
report,  however. 

Given  that  equipment  serial  numbers  133  and  93  have  decreasing 
failure  trends  or  for  that  matter,  if  an  equipment  displayed  an  increasing 
trend,  the  question  arises  as  to  the  use  of  the  field  failure  data  to 
p-.plain  it.  One  might  hypothesize,  for  the  decreasing  failure  trend  case, 
that  failures  traceable  to  poor  workmanship  (bad  solder  joints,  board  manu- 
facture, etc.)  or  a bad  lot  of  parts  could  have  occurred.  Failures  traceable 
to  such  causes  tend  to  fail  early  and  could  possibly  have  "escaped"  production 
test  screens.  On  the  other  hand,  an  increasing  failure  trend  for  an  equip- 
ment could  possibly  be  traceable  to  faulty  manufacture,  maintenance  practices 
or  to  field  environmental  conditions  which  tend  to  accelerate  failures.  A 
review  of  the  failure  data  contained  in  the  Equipment  R&M  Logs  for  each  of 
the  above  equipments,  however,  could  not  provide  any  clear  insights  into 
underlying  causes.  The  data  were  not  sufficiently  detailed,  nor  complete, 
for  performing  such  evaluations. 

Distribution  analysis  of  this  type  does  provide  a means  for 
selecting  particular  sequences  of  failures  for  more  detailed  investigations. 

For  example,  the  early  sequence  of  failures  which  occurred  on  equipment 
serial  numbers  133  and  93  (see  Figures  14  & 20)  would  be  good  candidates 
for  more  detailed  investigations  on  failure  causes.  Corrective  actions  on 
new  system  programs  could  then  be  taken  (i.e.,  improved  screening,  burn-in 
etc.)  depending  upon  the  findings  of  such  investigations. 


5.4.2  Goodness-of-Fi t Test  for  Exponents ali tv 


Since  there  was  not  any  significant  evidence  of  trend  in 
failure  data  for  10  of  the  12  equipments,  the  failure  times  were  ordered  by 
magnitude  for  each  equipment  and  goodness-of-fit  tests  were  applied.  Use 
of  the  exponential  model  was  the  most  logical  first  choice.  The  test  for 
exponential ity  which  was  applied  to  the  failure  data  is  given  in  reference 
(11)  below.  The  basic  computing  form  of  the  test  is: 


[ n “ ^(ti;  9),  F(  t-j ; 9)  - ] 


n 

Sn  = z 
i=l 


where  Sn  is  the  test  statistic 

A 

F (t-j,  9)  - failure  distribution  function  with  mean  equal  to 
n 

l ti 

9 = _Li! 


i = (1,  2,  ...  n)  is  the  number  of  failures  which  occur  in  t<ti 
n = total  number  of  failures 


Critical  values  of  Sn  are  tabulated  in  reference  11  for  various  levels  of 
significance.  The  hypotheses  of  exponential i ty  is  rejected  at  the  a% 
level  of  significance  if  Sn  is  larger  than  the  tabulated  value  of 


(11)  Mann,  N.R.,  Schafer,  R.E.,  Singpurwalla,  N.D.,  "Methods  of  Statistical 
Analysis  of  Reliability  and  Life  Data",  1974,  Pages  336-339. 
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TABLE  8.  RESULTS  OF  TEST  FOR  EXPONENTIALITY 


SERIAL  NUMBER 

N 

SN  (OBSERVED) 

SN  (CRITICAL  VALUES) 

a = .01 

a = .05 

85 

19 

2.41 

2.56 

2.16 

64 

29 

1.79 

3.04 

2.59 

83 

19 

1.24 

2.56 

2.16 

71 

17 

1.89 

2.46 

2.08 

49 

14 

1.74 

2.28 

1.92 

133 

19 

1.36 

2.56 

2.16 

57 

17 

1.50 

2.46 

2.08 

28 

17 

1.24 

2.46 

2.08 

26 

22 

1.24 

2.72 

2.29 

30 

24 

1.80 

2.83 

2.39 

69 

24 

1.82 

2.83 

2.39 

93 

13 

0.99 

2.19 

1.87 

As  the  data  in  Table  8 indicate,  the  hypothesis  of  exponentiality 
cannot  be  rejected  at  the  1%  level  for  all  12  equipments.  Equipment  serial 
number  85,  however,  can  be  rejected  at  the  5%  level.  A plot  of  the  fitted 
vs  empirical  reliability  function  for  equipment  serial  number  85  in  Figure 
23,  on  page  62  shows  a poor  fit  and  suggests  an  increasing  hazard  function 
for  the  failure  times.  The  Wei  bull  distribution  would  be  a good  next  choice 
for  use  in  a goodness-of-fit  test  for  this  equipment.  However,  for  the 
purposes  of  this  report,  the  analysis  was  not  attempted.  It  is  interesting 
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to  note,  that  although  the  chronologically  sequenced  failure  times  for  serial 
number  85  showed  no  evidence  of  trend,  the  ordered  failure  times  show  an 
operating  time  dependency.  In  addition,  although  equipment  serial  numbers 
133  and  93  showed  a trend  toward  decreasing  failure  frequency,  the  test  for 
exponential ity  did  not  result  in  rejection.  The  inappropriate  ordering  of 
the  failure  times  for  these  equipments  is  the  reason  for  the  apparent  anomoly. 
For  the  interested  reader,  further  discussion  of  similar  phenomena  are  pro- 
vided in  references  12  and  13. 

5.4.3  Confidence  Bounds  on  the  System  MTBF 

The  90%  confidence  bounds  on  the  mean  were  calculated  for 
each  equipment  on  the  basis  that  the  underlying  failure  distribution  is 
exponential.  Equipment  serial  numbers  85,  133  and  93  are  also  included, 
although  the  field  data  indicate  that  it  is  doubtful  that  their  failure 
distributions  are  exponential. 

Table  9 provides  the  results,  showing  the  operating  time, 

A 

number  of  failures  (N),  point  estimate  of  the  MTBF  (9)  and  90%  confidence 
bounds  on  the  mean  for  each  equipment. 


With  the  exception  of  equipment  serial  number  133,  the 
confidence  bounds  contain  both  the  specified  and  predicted  values  of  the  MTBF, 
thus  indicating  that  the  reliability  program  conducted  for  the  AN/FYQ-47 
during  development  and  production  was  very  effective  in  meeting  contractual 
reliability  requirements. 
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TABLE  9.  90%  CONFIDENCE  BOUNDS  ON  THE  SYSTEM  FIELD  MTBF 


SERIAL  NUMBER 

OPERATING  HOURS 

N 

/>> 

9 

90%  CONFIDENCE 
BOUNDS 

85 

13,499.1 

19 

710.5 

474.7-1178.7 

64 

14,299.0 

29 

493.1 

353.4-  735.0 

83 

11,923.7 

19 

627.6 

419.3-1041.2 

71 

12,797.1 

17 

752.8 

492.9-1289.5 

49 

10,977.4 

14 

784.1 

493.8-1434.2 

133 

6,409.3 

19 

337.3 

225.4-  559.7 

57 

6,962.7 

17 

409.6 

268.2-  701.6 

28 

10,360.3 

17 

609.5 

399.0-1043.9 

26 

12,947.8 

22 

588.5 

403.5-  938.2 

30 

12,824.3 

24 

534.3 

371.7-  833.5 

69 

11,601.2 

24 

483.4 

336.2-  753.0 

93 

8,525.0 

13 

655.8 

397.1-1202.4 

' 5.4.4  Reliability  Functions  - Fitted  vs  Empirical 

In  order  to  illustrate  the  fit  of  the  fail). re  data  to  the 
exponential  reliability  function,  plots  of  the  data,  showing  the  fitted  vs 
empirical  reliability  functions,  were  prepared  for  equipment  serial  numbers 
26,  28  and  85  in  Figures  21,  22  and  23  respectively.  Equipment  serial 
numbers  26  and  28  in  Figures  21  and  22  show  a good  fit  and  corroborate  the 
results  of  the  tests  for  exponential ity.  As  discussed  earlier,  the  plot 
of  the  data  for  equipment  serial  number  85  in  Figure  23  crosses  the  empirical 

U 
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reliability  function  only  once  from  above,  thus  suggesting  an  increasing 
failure  rate.  Figure  24  contains  the  pooled  data  for  all  of  the  equipments 
As  can  be  noted,  the  fit  to  the  exponential  is  excellent. 


EQUIPMENT  SERIAL  NO.  26 


EQUIPMENT  SERIAL  NO.  28 


FIGURES  21  & 22.  FITTED  VS  EMPIRICAL  EXPONENTIAL 
RELIABILITY  FUNCTIONS 


5.5  Summary  of  Findings 

(a)  Time-to- failure  data  are  obtainable  from  the  field  data  for 
ground  electronic  equipment  operating  in  mission  environments  similar  to  the 
AN/FYQ-47.  Records  of  operating  time  must  be  maintained  by  the  R&M  data 
collection  agency,  from  the  date  of  initial  installation  of  the  equipment  in 
the  field. 

(b)  Analysis  of  the  field  failure  distribution  characteristics  of 
an  equipment  provides  a very  useful  tool  for  evaluating  the  effectiveness  of 
the  reliability  assurance  techniques  used  during  design,  development  and 
production . 

(c)  It  is  essential  that  procedures  be  established  for  determining 
root  causes  for  selected  failures,  in  order  for  the  distribution  analyses  to 
be  fully  effective  in  yielding  corrective  actions. 

(d)  Particular  sequences  of  failures  can  be  selected  for  more 
detailed  investigation  of  causes  based  upon  the  distribution  analyses. 
Increasing  or  decreasing  failure  frequency  trends  can  be  used  as  the  basis 
for  selecting  failures  for  determination  of  causes  and  corrective  actions. 

(e)  Serial  number  tracking  of  removed  and  replaced  subassemblies 
is  not  normally  done  under  current  data  system  procedures.  Although  such 
tracking  would  be  required  for  some  types  of  analyses,  the  amount  of  record 
keeping  involved  would  be  prohibitive.  Subassembly  serial  number  tracking, 
therefore,  should  be  done  on  a sampling  basis  to  meet  specifically  defined 
reliability  analysis  objectives. 


(f)  To  ensure  timely  identification  of  failure  distribution 
trends,  computer  routines  including  trend  tests,  failure  distribution 
plotting,  goodness-of-fit  tests,  etc.,  should  be  implemented. 


6.0  FIELD  MAINTAINABILITY  DATA  ANALYSES 


The  downtimes  recorded  in  the  R&M  Logs  for  each  maintenance  action  were 
extracted  from  the  AFM  65-110  (ESR)  data  records.  Provisions  for  recording 
the  start  and  stop  times  when  the  equipment  is  down,  as  well  as  coded 
information  pertaining  to  the  reasons  for  the  downtime  are  included  in  the 
ESR  data.  Thus,  it  is  possible  to  separate  the  downtime  due  to  maintenance 
and  administrative  or  supply  delays.  In  compiling  the  data  for  the  analyses, 
the  downtimes  for  reasons  other  than  for  equipment  maintenance  were  excluded. 
The  maintenance  downtimes,  as  reported  in  the  data  records,  were  then  used 
as  the  basic  active  repair  time  for  each  maintenance  action.  There  is  no 
question  that  some  "hidden"  administrative  or  supply  delay  time  tends  to 
inflate  the  reported  maintenance  (repair)  times.  For  example,  a substantial 
number  of  repair  times  was  greater  than  eiqht  hours  in  duration.  It  is 
difficult  to  envision  active  repair  times  of  15-20  hours  which  do  not  include 
some  delay  times.  Possibly  the  technician  in  reporting  the  data,  may  have 
found  it  difficult  to  separate  active  repair  time  from  administrative  or 
supply  delays  (e.g.,  coffee  breaks,  waiting  for  spare  parts,  shift  changes, 
etc.).  In  any  event,  the  data  provided  no  bases  or  rationale  for  censoring 
"abnormally  long"  repair  times.  Even  if  the  long  repair  times  were  accurately 
reported,  there  are  no  procedures  in  the  data  system  for  reporting  the  reasons 
why  such  long  repair  times  occurred.  In  the  repair  time  distribution  analyses 
of  the  next  section,  the  "as  reported"  repair  times  are  used  with  all  re- 
ported delay  time  excluded. 

6.1  Field  Time- to- Repair  Distribution 


The  Lognormal  Probability  Distribution  is  often  used  to 


characterize  the  repair  time  distribution  for  electronic  equipment.  By 
plotting  the  field  repair  time  data  on  Lognormal  Probability  Paper,  a 
subjective  assessment  of  the  "fit"  of  the  distribution,  as  well  as  estimates 
of  the  distribution  parameters  can  be  obtained.  Figure  25  provides  a plot 
of  the  repair  times  for  the  AN/FYQ-47  malfunctions  which  were  listed  in 
the  AFM  65-110  data.  The  plotted  data  show  a good  fit  to  the  lognormal 
distribution.  Adjustment  and  preventive  maintenance  actions  were  not  in- 
cluded in  the  data  plot  since  they  were  representative  of  a different  task 
population.  These  data  were,  however,  investigated  in  a separate  analysis 
(not  shown)  without  very  much  change  in  the  results.  The  following  parameter 
estimates  were  obtained  from  the  graphical  analysis. 

TABLE  10.  AN/FYQ-47  FIELD  MAINTAINABILITY  PARAMETER  ESTIMATES 


PARAMETER 

HOURS 

Mean-Time-To-Repair  (MTTR) 

3.9 

98th  Percentile 

28.0 

Med ian-Time-To- Repair 

1.2 

6.2  Comparison  with  Maintainability  Requirements 

The  specified  maintainability  retirements  for  the  AN/FYQ-47  were 
a mean-time- to-repair  of  .5  hours,  a maxi  mum- time- to- repair  (98th  percentile) 
of  two  hour'  with  no  repair  takinq  greater  than  eight  hours.  For  comparison 
purposes,  the  "requirement"  distribution  is  shown  as  the  dotted  line  in 
Figure  25. 
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FIGURE  25.  FIELD-TIME- TO-REPAIR  LOGNORMAL  DISTRIBUTION 

There  is  obviously  a significant  difference  between  the  contractual  main- 
tainability requirements  and  the  field  maintainability  estimates.  The  data 
does  not  provide  sufficient  information  to  explain  the  disparity.  A recently 


completed  RADC  contractual  study  entitled,  "Operational  Influences  on 
Maintainability"^4),  examined  the  reasons  for  the  large  difference  between 
predicted  and  field  maintainability  estimates.  A large  percentage  of  the 
difference  was  found,  in  the  referenced  study,  to  be  traceable  to  operational 
factors  such  as,  lack  of  proper  tools,  poor  lighting,  sparing  inadequacies, 
changes  in  maintenance  policies,  and  technical  manual  ambiguities  which  tend 
to  inflate  the  active  repair  time.  Some  of  the  observed  long  repair  times 
investigated  in  the  referenced  study  were  found  to  be  the  result  of  inter- 
mittent failures  in  the  equipment  and  in  failures  of  mechanical  components. 
Degradation  in  technician  proficiency  caused  by  the  relatively  infrequent 
need  for  maintenance  was  also  found  to  be  parti?. ly  responsible  for  the 
differences. 


Obtaining  good  quality  maintainability  data  from  the  field  may  be 
more  difficult  than  obtaining  reliability  data.  It  would  be  necessary  to 
have  accurate  reporting  of  corrective  maintenance  time  broke,n  down  into  its 
subelements,  e.g.,  preparation,  fault  location,  fault  correction,  and  check- 
out times.  Such  reporting,  coupled  with  possibly  additional  coded  information 
indicating  the  reasons  why  repair  time  elements  took  longer  than  some 
established  nominal  time,  would  be  required.  In  this  manner,  analyses  of  the 
data  from  different  sites  would  enable  isolating  the  causes  for  poor  maintain- 
ability. Effectiveness  of  built-in-test  equipment  could  be  evaluated  by 
observing  the  distribution  of  fault  diagnosis  times  and  by  evaluating  the 
reasons  for  large  departures  from  expected  values.  The  distribution  of 
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fault  correction  times  can  be  similarly  used  to  evaluate  the  equipment 
design  in  terms  of  accessibility,  modularity,  etc.,  as  well  as  the  adequacy 
of  technical  manuals,  tools,  etc.  Once  the  causes  of  poor  maintainability 
are  isolated,  then  corrective  actions  in  maintainability  prediction  and 
demonstration  techniques  for  new  system  programs  can  be  taken.  Air  Force 
field  maintenance  data  are,  however,  currently  inadequate  for  performing 
such  evaluations. 

6.3  Summary  of  Findings 

(a)  Active  repair  time  for  performing  corrective  maintenance  is 
obtainable  from  the  field  data  by  subtracting  administration  and  supply 
delay  times  from  the  total  downtime.  However,  to  minimize  the  extent  to 
which  the  active  repair  time  is  contaminated  with  delay  time,  special  report- 
ing instructions  and  procedures  should  be  implemented  in  the  field. 

(b)  Even  if  the  active  repair  times  were  accurately  reported, 
there  are  no  provisions  in  the  data  system  for  reporting  the  reasons  why 
abnormally  long  repair  times  occur.  Improvements  can  be  made  in  the  procedures 
by  specifying  some  nominal  time  (e.g.,  represented  by  the  90th  percentile) 
beyond  which  the  reason  for  the  long  repair  time  should  be  reported.  Block 

27  of  the  AFTO  349  form  (See  Appendix  A)  can  be  used  for  this  purpose  or  a 
coding  system  could  be  used. 

(c)  Currently  there  are  not  any  procedures  in  the  data  system  for 
reporting  corrective  maintenance  time  broken  down  into  its  subelements  (e.g., 
fault  location,  fault  correction,  etc.).  Collection  of  such  data,  however. 


would  be  best  done  on  a sampling  basis  to  meet  a specific  analysis  objective. 


(d)  Analysis  of  the  repair  time  distribution  provides  a very  use- 
ful tool  for  evaluating  the  accuracy  and  effectiveness  of  the  maintainability 
assurance  techniques  used  during  the  equipment  design  and  development. 

(e)  Equipment  active  repair  time  data  reported  from  different 
sites  can  be  used  effectively  for  isolating  maintainability  problem  areas. 

Net  only  are  accurately  reported  active  repair  times  necessary,  but  in 
addition,  accurate  knowledge  of  failure  frequency  is  also  required. 


7.0  CONCLUSIONS  AND  RECOMMENDATIONS 


Recently,  there  have  been  proposals  and  recommendations  for 
improving  the  quality  of  Air  Force  field  maintenance  data  for  use  in  field 
R&M  assessments.  The  proposals  have  ranged  from:  (a)  scrapping  entirely  the 
AFM  66-1  system  and  starting  anew,  (b)  modifying  the  data  collection  forms 
and  procedures  to  include  needed  R&M  data  elements,  (c)  using  special  dedicated 
R&M  data  collection  "tiger  teams"  at  the  field  sites,  and  (d)  implementing 
"real-time"  computer  processing  of  the  maintenance  data,  coupled  with  remote 
access  and  query  capability  by  user  agencies.  Although  improvements  have 
been  made  in  the  data  systems  as  a result  of  some  of  the  recommendations, 
there  remains  a need  for  good  quality  field  R&M  data. 

Underlying  the  conclusions  and  recommendations  to  be  made  as  a result  of 
this  study  are  three  basic  premises.  First,  Air  Force  maintenance  and  status 
reporting  systems  will  continue  to  be  used  in  their  present  form.  It  should 
be  recoqnized  that  these  data  systems  were  not  designed  for  R&M  analysis 
purposes.  There  are  many  primary  users  of  the  data  (logistics  and  maintenance 
managers)  who  do  not  have  the  same  analysis  objectives  as  the  R&M  community 
does.  Second,  Air  Force  maintenance  technicians  are  primarily  motivated  to 
repair  the  equipment  and  not  to  discern  root  causes  of  failure  when  they  are 
confronted  with  competing  failure  cause  circumstances.  In  addition,  field 
technician  personnel  are  already  burdened  with  large  amounts  of  paperwork  and 
are  justifiably  motivated  to  "fix"  the  equipment  and  not  to  document  failure 
details.  As  a result,  failure  data  reported  from  the  field  tends  to  be 
"muddy"  at  best,  and  in  need  of  validation.  Third,  it  is  more  desirable. 


if  not  essential,  to  have  good  quality  R&M  data  on  a few  selected  systems, 
than  the  mass  of  generally  incomplete  and  fragmented  R&M  data  which  currently 
exists  for  most  fielded  systems. 

7.1  Concl usions 

(a)  One  of  the  major  conclusions  of  the  study  is  that  the  use  of 
Air  Force  maintenance  data  systems,  coupled  with  the  information  provided 
in  TOs,  Work  Unit  Code  Manuals,  etc.,  provide  much  of  the  data  needed  to 
perform  field  R&M  assessments.  In  fact,  the  levels  of  analysis  detail 
attained  in  this  study,  go  much  further  than  what  was  thouqht  possible. 
However,  the  data  systems  fall  short  in  certain  key  areas,  which  tends  to 
degrade  the  potential  usefulness  of  the  data  for  achieving  R&M  analysis 
objectives.  In  addition,  unless  a way  is  found  to  correct  the  deficiencies, 
further  developments  and  use  of  this  very  valuable  R&M  data  source  are  not 
likely  to  occur. 

(b)  The  most  serious  failing  of  Air  Force  data  systems  for  use  in 
reliability  assessment  is  the  lack  of  primary  failure  cause  information 
identified  to  the  lowest  level  (part)  of  assembly.  The  reason  for  the 
former  condition  stems  partially  from  incomplete  reporting  of  failure  data 
from  the  field.  However,  even  if  the  reporting  were  reasonably  complete, 
there  would  still  be  a need  for  verifying  the  failure  circumstance  data  to 
ascertain  that  a bonafide  system  failure  had  occurred  and  that  the  primary 
cause  has  been  properly  identified. 

(c)  Laboratory  failure  analysis  of  selected  failed  parts  is  re- 
quired to  further  pinpoint  failure  causes  so  that  analysis  objectives  might 
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be  fully  achieved,  i.e.,  responsibility  for  the  failure  is  assigned  and 
corrective  actions  determined. 

(d)  The  reliability  design  prediction  and  the  reliability  model 
are  essential  elements  of  the  field  reliability  assessment  program.  Use  of 
the  model  enables  a proper  accounting  of  the  field  failures  to  be  made. 

The  predictions  provide  a baseline  against  which  field  failure  rates  can  be 
compared.  Without  the  prediction  data,  items  which  have  abnormally  high  or 
low  failure  rates  cannot  be  properly  identified.  In  those  cases  where  the 
prediction  data  may  be  suspect,  comparisons  of  failure  rates  can  be  made 
for  the  same  item  used  in  different  equipment  or  circuit  applications.  Thus, 
use  of  the  prediction  data  also  provides  a basis  for  modifying  prediction 
procedures  which  might  have  resulted  in  overly  optimistic  or  pessimistic 
failure  rates. 

(e)  The  equipment  technical  orders,  work  unit  code  manuals,  and 
illustrated  parts  breakdown  are  also  required  for  the  field  R&M  assessment 
program.  Hardware  identifications  which  are  made  between  the  early  design 
and  field  deployment  are  evolutionary  in  nature.  As  a result,  there  are 
part  numbers  assigned  by  the  equipment  manufacturer,  generic  part  identifica- 
tions assigned  by  part  vendors,  and  work  unit  codes  assigned  by  the  govern- 
ment. Resolving  hardware  identification  ambiguities  in  the  data  is  one  of 
the  most  time-consuming  and  difficult  tasks  in  performing  field  R&M  evalua- 
tions. The  use  of  part,  board,  and  assembly  cross  identification  lists  are 
required  to  edit  and  validate  the  data  prior  to  processing  and  analysts. 

(f)  Special  data  reporting  and  processing  procedures  are  required 
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for  those  systems  for  which  R&M  assessments  are  desired.  Said  procedures 
should  remain  within  the  framework  of  the  current  data  systems  so  as  not  to 
result  in  additional  paperwork  for  the  field  technician  nor  cause  disruption 
in  existing  data  processing  programs  and  to  other  data  system  users. 

(g)  R&M  data  collected  from  the  field  must  be  supplemented  with 
additional  inputs  developed  by  the  group  or  agency  which  is  performing  the 
data  collection  and  analyses. 

7.2  Recommendations 

(a)  A sampling  plan  should  be  developed  for  those  equipments  in 
the  field  for  which  R&M  assessments  are  desired.  The  plan  should  be  based 
upon  such  factors  as:  the  total  population  of  equipments  in  the  field, 
expected  failure  frequency,  equipment  part  count  and  the  need  for  obtaining 
specific  information  pertinent  to  defined  analysis  objectives. 

(b)  The  equipments  to  be  sampled  in  the  field  should  be  "red 
flagged"  so  that  field  personnel  know  that  special  data  collection  and 
handling  procedures  will  be  in  effect  for  a specified  period  of  time. 

(c)  An  AFSC  Supplement  to  AFM  66-1  and  AFM  65-110  should  be  prepared 
to  augment  data  collection  and  handling  procedures  for  those  equipments  to 

be  sampled  for  field  R&M  assessments. 

(d)  The  AFSC  supplement  should  provide  procedures  and  guidance  for 
the  special  reporting  and  handling  of  field  R&M  data  which  include: 

(1)  Emphasis  on  complete,  accurate  and  timely  reporting  of 
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all  failure  circumstance  data  on  existing  data  collection  forms. 


» 


(2)  The  need  for  accurate  identification  of  primary  failure 
cause  information  to  the  lowest  level  of  assembly. 

(3)  The  use  of  narrative  descriptions  of  the  failure  and 
repair  action  in  the  blocks  provided  on  the  existing  forms 

(4)  The  accurate  use  of  coded  information  (additional  codes, 
when  discovered  codes,  action  taken  codes)  on  the  forms  so  that  proper 
accounting  of  the  failures  can  be  made. 

(5)  Provisions  for  obtaining  a copy  of  the  raw  data  forms 
from  the  field  on  a timely  basis. 

(6)  Provisions  for  obtaining  failed  devices  from  the  field 
for  failure  analyses. 

(7)  Procedures  for  recording  coded  maintainability  information, 
in  the  AFM  65-110  data,  pertaining  to  subtask  times,  (fault  location,  fault 
correction,  preparation,  etc.)  of  corrective  maintenance  time.  In  addition, 
procedures  for  recording  the  reason  for  "abnormally  long"  repair  times  (e.g., 
greater  than  the  specified  90th  percentile)  should  be  provided  in  the 
supplement. 

(e)  Conmunication  procedures  (TWX,  telecopier,  etc . ) between 
supervisory  field  maintenance  personnel  and  the  group  or  agency  performing 
the  R&M  analyses  should  be  established. 


(f)  Each  of  the  failures  reported  from  the  field  should  be 
validated  and  verified  by  the  group  responsible  for  performing  the  analyses 
prior  to  data  processing.  Validation  would  ensure  that  correct  hardware 
identification  and  complete  failure  documentation  has  been  made.  Verification 
implies  identifying  that  a bonafide  failure  has  occurred  and  that  the  correct 
primary  cause  has  been  determined.  Such  verification  should  perhaps  best  be 
done  on  as  near  a real  time  basis  as  possible  through  communication  where 
necessary  with  field  personnel.  The  alternative  to  this  approach  is  to 
attempt  to  reconstruct  failure  circumstances  which  might  have  occurred  3-6 
months  in  the  past. 

(g)  Computer  processing  of  the  validated  and  verified  R&M  data 
should  be  done  with  provisions  for  updating,  with  additional  information 
(e.g.,  failure  analysis  results)  which  may  become  available  later. 

(h)  Computerization  of  analysis  tasks  such  as  calculation  of 
expected  failures,  trend  tests,  goodness-of-fi t tests,  plotting  of  failure 
data,  etc.,  should  be  done  so  as  to  facilitate  timely  preliminary  analysis 
and  the  selection  of  failed  parts  for  more  detailed  investigation  of 
failure  causes. 

(i)  R&M  program  data  (prediction,  demonstration,  etc.),  the 
parts  count  and  the  technical  and  work  unit  code  manuals  should  be  available 
for  the  R&M  assessment  program. 

Fiqure  26  illustrates  the  various  elements  and  the  data  flow  inherent 
to  the  recommendations  made  above. 
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APPENDIX  A 


A.l  AIR  FORCE  f MAINTENANCE  DATA  COLLECTION 

Data  collection  procedures  are  the  key  area  in  any  data  system  since 
they  provide  the  means  by  which  the  data  are  initially  entered  into  the 
system.  If  specific  data  elements  which  are  required  for  R&M  assessment 
are  not  collected,  then  obviously,  no  amount  of  machine  processing  or 
manipulation  of  the  data  will  provide  the  desired  results.  The  complexity 
of  the  R&M  data  collection  task  is  staggering  when  one  considers  the  variety, 
use  conditions  and  numbers  of  equipment  in  the  Air  Force  inventory.  It  is 
obvious  that  the  collection  of  R&M  data  for  an  indefinite  period  and  for  all 
the  equipment  in  the  inventory  is  certainly  not  a desirable  nor  practical 
goal.  A controlled  R&M  data  collection  on  a limited  number  of  selected 
equipments  for  a sufficient  time  to  obtain  adequate  sample  data  is  a much 
more  reasonable  alternative. 

The  ground  rules  for  the  collection  of  data  under  the  AFM  66-1  and 
AFM  65-110  systems  differ  considerably  depending  upon  whether  the  equipment 
is  part  of  a ground  CEM,  aircraft  or  missile  system.  The  using  command  in 
which  an  equipment  is  located,  and  the  maintenance  policy  under  which  it 
is  maintained  also  have  an  effect  on  the  type  and  quality  of  the  data  which 
are  collected.  For  example,  when  repairs  are  accomplished  at  a depot  or 
some  off-base  location,  the  problem  of  traceability  in  the  failure  data  to 
lower  equipment  levels  is  certainly  more  difficult,  if  not  impossible  to 
achieve.  The  problem  of  failure  and  operating  time  data  collection  is  much 
more  complicated  for  an  avionic  equipment  which  may  be  operated  periodically 


procedures  are  divided  into  two  categories  identified  as  on-equipment  and 
off-equipment  maintenance.  On-equipment  maintenance  consists  of  maintenance 
actions  on  complete  end-items  of  equipment  (e.g.,  AN/FYQ-47) . Remove  and 
replace  actions,  adjustments  and  calibrations  are  reported  as  on-equipment 
actions.  In-shop  maintenance  actions  involving  replacement  of  parts  on  sub- 
assemblies  or  circuit  boards  are  identified  as  off-equipment  maintenance. 

It  is  important  to  note,  that  since  the  AFM  66-1  is  a base  level 
maintenance  management  system,  little  provision  is  made  for  recording  off- 
equipment  maintenance  data  for  repairs  accomplished  at  a depot,  special 
repair  agencies  or  by  a contractor.  Thus,  the  traceability  of  the  failure 
to  lower  assembly  levels  is  often  lost,  or  at  least  much  more  difficult, 
when  off-base  repair  is  performed.  The  maintenance  policy  for  the  AN/FYQ-47, 
however,  entails  on-site  repair  to  the  part  level  so  that  off-base  traceability 
was  not  a problem  in  this  case. 

For  on-equipment  work  on  ground  CEM  equipment  the  primary  entries 
on  the  AFTO  Form  349  are  shown  in  the  unshaded  blocks  of  Figure  Al.  For 
off-equipment  work  on  removed  boards  assemblies,  etc.,  the  same  unshaded 
blocks  are  used  in  addition  to  blocks  19  through  21  and  the  reverse  side 
of  the  Form  Block  29  (see  Figure  A2).  The  AFTO  Form  350,  (not  shown) 
is  a two-part  form  that  is  attached  to  boards  or  components  that  are  removed 
during  on- equipment  maintenance.  The  form  serves  as  an  identification  and 
status  tag  which  provides  the  information  required  in  completing  the 
AFTO  349  for  off-line  work. 
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FIGURE  A2.  AFM  66-1  MAINTENANCE  DATA  COLLECTION  RECORD 
(REVERSE  SIDE  AFTO  349) 

A description  of  the  data  elements  of  the  AFTO  Form  349  is  given 
below: 

Block  1 - Job  Control  Number  (JCN):  The  JCN  consists  of  seven 
characters,  the  first  three  characters  are  the  Julian  date,  and  the  last 
four  are  used  to  identify  jobs  which  are  assigned  a monthly  job  sequence 
number  (e.g.,  the  first  job  of  the  month  occu^ng  on  10  Feb  would  have 


(MDS)  Equipment  Classification  Code:  These  blocks  are  used  to  identify 
the  equipment  on  which  work  was  performed. 


Block  6 - Time:  The  downtime  in  hours  and  tenths  of  hours  that 


the  item  identified  in  Block  3 was  not  available  for  use. 


Type  Maintenance  Code:  The  type  maintenance  code 


the  JCN  0410001).  The  definition  of  a "job"  falls  into  four  basic  categories 
of  work,  equipment  malfunctions,  inspections,  support  general  work  and  time- 
change  requirements.  Equipment  failures  which  require  unscheduled  corrective 
maintenance  are  assigned  a JCN.  The  JCN  provides  a means  to  tie  together 
all  on  and  off-equipment  maintenance,  including  all  items  replaced  on-line 
to  correct  the  failure  and  the  part  replacements  occurring  during  off- 
equipment  maintenance  of  removed  items. 

Block  2 - Work  Center:  The  work  center  is  used  to  identify 
organizational  elements  to  which  maintenance  personnel  are  assigned  or 
locations  to  which  they  may  be  dispatched. 

Blocks  3-5  - Identification  (ID)  Number,  Missi on-Des iqn-Ser i es 


consists  of  one  character  and  is  used  to  identify  the  type  of  work  that 
was  accomplished  such  as  scheduled  or  unscheduled  maintenance. 


Column  C - Work  Unit  Code:  The  work  unit  code  consists  of  five 


characters,  and  is  used  to  identify  the  system,  subsystem  or  circuit  card 
on  which  maintenance  is  required.  The  first  two  positions  of  the  work  unit 
code  identify  the  functional  system  (e.g.,  CA000  represents  the  FYQ-47 
Central  Processing  System).  The  third  and  fourth  positions  identify  sub- 
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systems  or  major  assemblies  as  applicable  (e.g.,  CAAOO  represents  the  OL-46/ 
FYQ-47  Data  Programming  Group,  CABOO  represents  theAN/FYA-83  Indicator 
Group,  and  CAAAO  represents  the  Height  Finder  Module).  The  fifth  position 
of  the  work  unit  code  normally  identifies  repairable  items  (e.g.,  CAAAA 
represents  the  8-Bit  Ladder  Board).  The  work  unit  codes  are  published  in 
work  unit  code  manuals  for  each  weapon  and  support  system. 

Column  D - Action  Taken:  The  "Action  Taken"  code  consists  of  one 
character  and  is  used  to  identify  the  maintenance  action  that  was  taken  such 
as  remove  and  replace.  A list  of  authorized  "Action  Taken"  codes  is  con- 
tained in  the  work  unit  code  manual  for  the  equipment. 

Block  E - When  Discovered  Code:  The  "When  Discovered"  code 
consists  of  one  character  and  is  used  to  identify  when  a failure  was  dis- 
covered such  as  during  inspection  or  equipment  operation.  A list  of 
authorized  "When  Discovered"  codes  is  contained  in  the  work  unit  code  manual 
for  the  equipment. 

Block  F - How  Malfunctioned  Code:  The  "How  Malfunctioned"  code 
consists  of  three  characters  and  is  used  to  identify  how  the  equipment, 
assemblies  or  board  malfunctioned.  A complete  list  of  "How  Malfunctioned" 
codes  are  contained  in  the  work  unit  code  manual  for  the  equipment. 

Columns  H and  I - Start/Stop  Day/Hour:  The  day  and  hour  (to  the 
nearest  five  minutes)  that  the  job  described  on  the  line  was  started  and 
stopped. 


1 

Column  J - Crew  Size:  The  number  of  personnel  involved  in  the 
job  described  on  the  line. 

Column  N - Man  No. : A five  position  locally  assigned  man  number 
consisting  of  the  first  letter  of  his  last  name  in  the  first  position  and 
the  last  four  digits  of  his  social  security  number. 

Blocks  26  and  27  - Discrepancy  and  Corrective  Action:  A descrip- 
tion of  the  discrepancy  or  work  to  be  performed  and  the  action  taken  to 
correct  the  problem.  Provision  is  made  for  continuation  of  any  narrative 
in  Block  30  on  the  reverse  side  of  the  form. 

Block  29  - Parts  Replaced  During  Repair:  This  block  is  used  to 
record  the  identification  of  parts  replaced  during  repairs.  Specific  entries 
are  outlined  below: 

Line  Number  Column:  The  line  number  from  the  front  side  of  the 
form  that  identifies  the  Item  to  which  the  parts  replacement  data  recorded 
in  Column  H through  F of  Block  29  is  related. 

Column  A - Federal  Supply  Class:  The  Federal  Supply  Class  Code 
which  applies  to  the  part  being  replaced. 

Column  B - Part  Number:  The  manufacturer's  part  number  of  the 
part  being  replaced. 

Column  C - Work  Unit  Code:  The  work  unit  code  of  the  replaced 
part  (If  assigned).  If  the  replaced  part  does  not  have  an  assigned  work 
unit  code  the  next  higher  assembly  work  unit  code  is  entered. 
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FIGURE  A4.  AFM  65-110  JOB  CONTROL  DOCUMENT  (AF  FORM  2445) 

Listed  below  are  descriptions  of  the  data  elements  contained  on  the  forns. 

Card  Identifier  (Column  1):  The  card  identifier  is  used  to 
Identify  each  of  the  three  types  of  data  which  are  reported,  that  is, 
equipment  status/utilization  reports,  inventory  change  cards  and  delay  or 
text  trailer  cards. 

Major  Command  and  Subcommand  Code  (Column  2-3):  These  columns 
identify  the  major  command  and  subcommand  to  which  the  reporting  unit  belongs. 
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Local  Use  (Column  4-14):  These  columns  are  available  for  use 
locally  by  the  reporting  units. 

ESR  No.  (Columns  15-22):  The  ESR  number  consists  of  seven 
characters,  the  first  three  numbers  are  the  Julian  date  and  the  last  four 
are  a locally  assigned  number  beginning  each  day  with  0001.  Column  15  is 
used  to  modify  the  JCN  on  the  AFTO  349. 

Organizational  Number  (Columns  23-26):  The  applicable  code  which 
identifies  the  site/organization  at  which  the  equipment  is  located. 

Subcommand  Number  (Columns  27  & 28):  The  subcommand  number  is 
entered  to  identify  the  subcommand  to  which  the  unit  is  assigned. 

Type  Model  Series  (Columns  29-36):  The  identification  of  the 
equipment  being  reported  on. 

Functional  Position  (Columns  37-38):  The  functional  position  is 
used  to  identify  an  individual  item  where  more  than  one  item  is  located  at 
a single  reporting  unit. 


Channel  (Column  39):  "ii i s data  element  is  used  to  identify 

redundant  channels  in  dual  channel  equipment  and  to  identify  other  major 
reportable  components  in  complex  end- items. 


Downtime  Code  (Column  40):  The  downtime  code  consists  of  one 
character  and  is  used  to  indicate  the  reason  for  the  downtime. 


Month  (Columns  41-42):  The  month  in  which  the  status  change 

occurred. 
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Stop  (Columns  43-48)  Start  (Columns  49-54):  The  date  and  time 
(in  hours  and  minutes)  that  the  status  change  incident  was  started  and 


stopped. 

Example:  An  equipment  went  down  at  0815  on  the  22nd  day  of  the 
month  and  was  returned  to  operation  at  0900.  Entries  would  be  made  as 
follows:  220815  and  220900  in  the  appropriate  columns. 

Duration  (Columns  55-58):  The  duration  of  the  downtime  calculated 
in  hours  and  tenths. 

Delay  Code  (Column  59):  This  data  element  is  used  to  identify 
the  nature  and  length  of  any  delay  encountered  in  returning  an  equipment 
or  channel  to  an  operational  condition. 

Multiple  Delay  Code  MDC  (Column  60):  The  sequential  order  of 
delays  in  returning  an  equipment  to  operational  condition  is  identified 
by  use  of  the  multiple  delay  code. 

Special  Code  (Column  61):  This  special  code  is  used  to  identify 
continuation  cards  of  active  incidents  which  are  carried  over  from  a previous 
reporting  period.  It  is  also  used  as  a code  to  identify  continuation  cards 
where  an  ESR  has  been  closed  and  reopened  for  some  other  reason. 

Work  Unit  Code  (Column  62):  Used  to  identify  the  lowest  work 
unit  coded  assembly  which  caused  the  malfunction. 

Serial  Number  (Column  67-72):  Used  to  identify  a particular  end 
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Subsystem  Outages  (Columns  73-78):  These  entries  are  used  to 
indicate  the  effects  of  an  equipment  outage  on  the  unit's  capability  to 
perform  its  mission.  An  entry  in  these  columns  indicate  that  the  flow  of 
data  for  a particular  subsystem  has  been  interrupted.  For  example,  a 
failure  in  the  FYQ-47  can  interrupt  the  flow  of  data  in  three  subsystems 
search  (Column  73),  height  (Column  75)  and  SIF  (Column  76). 

Equipment  Condition  (Column  79):  Entries  in  this  column  are  used 
to  identify  the  operating  condition  of  the  equipment  being  reported  on.  If 
an  equipment  end  item  is  not  fully  operational,  but  is  still  able  to  provide 
some  usable  data,  an  "N"  is  entered  in  this  column.  If  the  equipment  end 
item  is  unable  to  perform  its  function,  a "G"  is  entered  in  this  column. 


JtM.  . 


Mission  Capability  (Column  80):  This  entry  is  used  to  identify 
the  effect  of  an  ESR  on  a reporting  unit's  capability  to  perform  its  mission. 
If  there  are  any  subsystem  outages  (entries  in  Columns  73-78)  an  "R"  is 
entered.  If  Columns  73-78  are  blank  an  "A"  is  entered. 
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APPENDIX  B 


REQUIRED  R&M  DATA  ELEMENTS 

Listed  in  Table  B1  are  the  R&M  data  elements  required  for  field 
R&M  assessment.  Data  elements  1 through  33  are  those  required  for  reliability 
assessment,  and  data  elements  34  through  39  are  the  additional  elements 
needed  for  maintainability  assessment. 

Not  all  of  the  listed  data  elements  are  directly  obtainable  from  the 
data  collection  forms,  but  many  can  be  obtained  through  use  of  the  work 
unit  code  and  technical  manuals.  Comments  are  provided  below  pertaining  to 
those  data  elements  which  can  be  obtained  in  this  manner. 

Some  of  the  listed  data  elements  are  contained  on  the  forms,  but  are 
not  processed  into  the  data  system.  These  data  elements  are  used  locally 
by  field  personnel  at  the  base  level.  Comments  regarding  these  data  elements 
are  also  provided  in  the  subsequent  discussions. 

The  data  elements  listed  with  an  asterisk  are  those  for  which  the 
agency  or  group  performing  the  R&M  analyses  must  provide  additional  input. 
These  data  elements  refer  to  the  verification  and  proper  identification  of 
the  primary  cause  of  the  failure.  Data  element  number  33  is  the  most 
seriously  lacking  piece  of  data  since  it  is  essential  for  reliability 
assessment. 

Some  of  the  required  data  elements  are  contained  in  existing  data 
forms,  but  procedural  changes  are  required  so  that  the  data  can  be  more 
effectively  used  for  R&M  assessments.  These  data  elements  are  identified 


TABLE  B1 . R&M  DATA  ELEMENTS 
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1.  Report  Number 

2.  Original  Report  Number 

3.  Reporting  Activity 

4.  Work  Center 

5.  System  Type  Model , Series 

6.  System  Serial  Number,  Manufacturer 

7.  Equipment  Type,  Model,  Series 

8.  Equipment  Serial  Number,  Manufacturer 
*9.  Failed  Item  Part  Number 

*10.  Failed  Item  Reference  Designation 

*11.  Failed  Item  Name 

*12.  Failed  Item  Manufacturer 

13.  Failed  Item  Serial  Number 

14.  Next  Higher  Assembly  Part  Number 

15.  Next  Higher  Assembly  Serial  Number 

16.  Next  Higher  Assembly  Name,  Manufacturer 

17.  Replacement  Part  Number,  Serial  Number 

18.  Replacement  Part  Manufacturer,  Reference  Designation 

19.  Date  of  Malfunction 

20.  Type  of  Maintenance  Code 

21.  Narrative  Description  of  Trouble  (How  Malfunctioned) 

22.  Narrative  Description  of  Repaid  Action  (Action  Taken) 

23.  Activity  during  which  failed  Item  discovered  (When  Discovered) 

24.  Effect  of  Failure  (mission  failure,  performance  degradation,  no  effect) 

25.  Total  Parts  Replaced 

26.  Total  Operating  Time  (System  or  Equipment) 

27.  Operating  Time-to-Fallure  (Equipment) 

28.  Operating  Time  Failed  Item/Installed  Item 

*29.  Failure  Verified  (Verified  or  denied  by  bench  check) 

*30.  Type  of  Failure  (primary  or  secondary) 

*31.  Analysis  Required  (Yes  or  No) 

*32.  Failure  Analysis  Report  Number  and  Date 

*33.  Failure  Cause-Failed  Item  (misapplication,  overstress,  vendor  fault,  etc.) 

34.  Oelay  Code 

35.  Downtime  Code 

36.  Crew  Size 

37.  Skill  Level  Maintenance  Technician 

38.  Start  Time  for  Maintenance 

39.  Stop  Time  for  Maintenance 
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Data  Elements  1.2:  The  intent  of  these  elements  is  to  provide 
a unique  identifier  for  a failure  incident  and  to  provide  for  traceability 
of  failure  circumstance  data  which  may  extend  over  a period  of  time.  Use 
of  the  Job  Control  Number  does  provide  a means  for  tracing  the  failure  data, 
but  proper  dating  of  various  maintenance  actions  is  sometimes  ambiguous. 
Improvement  can  be  made  by  a change  in  procedure. 

Data  Elements  3-16  (Except  13):  These  data  elements  are  either 
directly  provided  on  the  reporting  forms  or  can  be  obtained  through  use  of 
the  work  unit  code  and  technical  manuals.  Correct  identification  of  the 
failed  item  (data  elements  9-12)  is  the  responsibility  of  the  agency  or 
group  performing  the  analyses.  These  data  elements  also  relate  to  the 
data  elements  (29-33). 

Data  Elements  17,  18,  13,  28:  Use  of  these  data  elements  are 
required  under  special  circumstances  when  components  have  to  be  changed 
after  a specified  period  of  use  (e.g.,  high  power  tubes  or  serially 
controlled  items).  Blocks  19-25  on  the  349  form  provide  these  data. 

Data  Elements  19-25:  These  data  elements  are  directly  provided 
for  on  either  the  AF  Form  349  or  182.  Narrative  descriptions  (data 
elements  22-23)  are  not  normally  processed  into  the  data  system.  Collection 
of  the  raw  data  form  must  be  done  to  obtain  these  data. 

Data  Elements  26T  27:  Operating  time  and  time- to- failure 
Information  may  be  obtained  by  processing  the  outputs  of  the  AFM  66-1  and 
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AFM  65-110  data  jointly  in  a chronological  manner.  Maintaining  operating 
time  records  beginning  with  the  initial  installation  of  the  equipment  in 
the  field  is  required. 

Data  Elements  29-33:  These  data  elements  are  essential  to 
meeting  reliability  assessment  objectives.  The  agency  or  group  performing 
the  analyses  must  supply  and/or  validate  these  data. 

Data  Elements  34-39:  These  data  elements  are  required  for 
maintainability  assessments  and  are  directly  provided  on  existing  data 
collection  forms.  Procedural  changes  and  additional  codes  are  required, 
however,  so  that  the  data  can  be  used  more  effectively  for  maintainability 
analysis  purposes. 
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PART/BOARD  PREDICTED  FAILURE  RATES  AND  FIELD  REPLACEMENT  RATES 


PART  TYPE  QUANTITY  TOTAL  PART  TOTAL  # OF  OBSERVED  PART  PREDICTED  EXPECTED 

HOURS  x 106  REPLACE-  RFPL . RATE  FAILURE  FAILURES 

(T)  MENTS  x IQ"6  RATE  x 10'6 


TOTAL  I 6250  1060.542  140  I 0.132  75.7 


PART  TYPE  QUANTITY  TOTAL  PART.  TOTAL  # OF  OBSERVED  PART  PREDICTED  EXPECTED 

HOURS  x 106  REPLACE-  REPL.  RATE  FAILURE  FAILURES 

(T)  MENTS  x TO*6  RATE  x 10"6 

(Xp)  (XpT) 


PART  TYPE  QUANTITY  TOTAL  PART,  TOTAL  # OF  OBSERVED  PART  PREDICTED  EXPECTED 

HOURS  x 10b  REPLACE-  REPL.  RATE  FAILURE  r FAILURES 

(T)  MENTS  x 10-6  RATE  x 10"6 

(Xp)  (ApT) 


BOARD  TYPE  QUANTITY  TOTAL  BOARD  TOTAL  # OF  OBSERVED  PREDICTED  EXPECTED 

HOURS  REPLACE-  BOARD  FAILURE  FAILURES 

x 106  MENTS  REPL.  RATE  RATE  x 10-6 

(T)  x 10-6  (Ap)  (ApT) 
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BOARD  TYPE  QUANTITY  TOTAL  BOARD  TOTAL  # OF  OBSERVED  PREDICTED  EXPECTED 

HOURS  REPLACE-  BOARD  FAILURE  FAILURES 

x 106  MENTS  REPL.  RATE  RATE  x 10"6 

(T)  x 10-6  (Xp)  (XpT) 


1 706-5368)  9 TT527  0 068 
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